!standard 10.01.02 (03) 03-09-22 AI95-00217-06/04 !standard 10.01.02 (04) !standard 10.01.02 (08) !standard J.10 (00) !class amendment 03-02-04 !status work item 03-02-04 !status received 03-02-04 !priority High !difficulty Hard !subject Limited With Clauses !summary A new kind of with_clause is proposed. !problem Ada allows mutually recursive types to be declared only if they are all declared within the same library unit. This can force all of the major data structures of a program into a single library unit, which is clearly undesirable in some situations. The goal of the proposed feature is to allow mutual recursion among type declared in separate packages (types containing (pointers to) each other as components, and primitive operations with (pointers to) each other as parameters), and to do this in a way that doesn't place any undue restrictions on the programmer. !proposal A new kind of with_clause is added, of the form "limited with ;". This is called a "limited_with_clause". The old fashioned kind is called a "nonlimited_with_clause". There are two key features of a limited_with_clause such as "limited with X;": - It introduces no semantic dependence on X (and hence no elaboration dependence on X). The implementation may choose to elaborate X either before or after the current unit. - Instead of providing visibility to the named packages and their content, it provides visibility only to the package names, any packages nested within them, and a view of each type therein. The type view is incomplete, implying all the usual restrictions on incomplete types. If the type is tagged, the view is tagged incomplete. Thus, cyclic chains of with_clauses are allowed, so long as the chain is broken by at least one limited_with_clause. !wording Replace 3.10.1(10) by: A dereference (whether implicit or explicit -- see 4.1) is only allowed to be of an incomplete type T if it does not occur in the immediate scope of [the incomplete type] T and: o it occurs in the scope of a nonlimited_with_clause that mentions the library package in which that completion is declared; or o its expected type is the completion of T. Replace 4.1(9) by: If the type of the name in a dereference is some access-to-object type T, then the dereference denotes a view of an object. The nominal subtype of that view is the designated subtype T, unless T is an incomplete type, in which case the nominal subtype is the completion of T. Change the beginning of 8.3(20): The declaration of a library unit (including a library_unit_renaming_declaration) is hidden from all visibility except at places that are within its declarative region or within the scope of a nonlimited_with_clause that mentions it. The limited view of a library package is hidden from all visibility except at places that are within the scope of a limited_with_clause that mentions it but not within the scope of a nonlimited_with_clause that mentions it. Replace 8.4(5) by: A package_name of a use_package_clause shall denote a non-limited view of a package. Add after of 8.4(10): o A potentially use-visible declaration is not visible if the place considered is within the scope of a limited_with_clause which: - is more nested than the use_clause; and - mentions the package where the declaration occurs; and - is not more nested than a nonlimited_with_clause which names that package. Replace 8.5.3(3) by: The renamed entity shall be a non-limited view of a package. Insert somewhere in 10.1.1(26): A limited library package view declaration depends semantically upon the parent limited package view declaration. A library package declaration depends semantically upon its limited view. Replace 10.1.2(4) by: with_clause ::= limited_with_clause | nonlimited_with_clause limited_with_clause ::= limited with library_unit_name {, library_unit_name}; nonlimited_with_clause ::= with library_unit_name {, library_unit_name}; Add after 10.1.2(5): A with_clause is *more nested than* a context_item if the scope of the with_clause is (strictly) included in the scope of the context_item. Similarly, a with_clause is *more nested than* a use_clause occurring immediately within a declarative region, if the scope of the with_clause is included in the scope of the use_clause. Replace 10.1.2(6) by: A library_item is *named* in a with_clause if it is denoted by a library_unit_name in the with_clause. A library_item is *mentioned* in a limited_with_clause if it is named in the with_clause or if it is denoted by a prefix in the with_clause. Add after 10.1.2(8): A library_item mentioned in a limited_with_clause shall be a package_declaration[, not a subprogram_declaration, generic_declaration, generic_instantiation, or package_renaming_declaration]. A limited_with_clause shall not appear on a library_unit_body or subunit. A limited_with_clause which names a library_item shall not be: o part of the same context_clause as a nonlimited_with_clause which mentions the same library_item; or o more nested than a nonlimited_with_clause which mentions the same library_item. Add after 10.1.4(2): In addition, for each library_package_declaration in the environment, there is a conceptual declaration of a *limited view* of that library package. The limited view of a package contains: - A declaration of the limited view of each nested package_declaration. - For each type_declaration in the visible part, an incomplete_type_declaration with the same name, whose completion is the type_declaration. If the type_declaration is tagged, then the incomplete_type_declaration is tagged incomplete. The limited view of a package has the same identifier as the package_declaration. It is private or public according to the declaration of the corresponding library_package_declaration. There is no syntax for declaring limited package views, because they are always implicit. A limited package view is _not_ the declaration of a library unit (the library_package_declaration is). Nonetheless, the declaration of a limited package view is considered to be a library_item. A library_package_declaration is considered to be the completion of its limited view declaration. Add after 10.1.4(3): The mechanisms for adding a unit mentioned in a limited_with_clause within an environment are implementation defined. Add after 10.2(6): o If the limited view of a unit is needed, then the full view of the unit is needed. !discussion It is natural to think of a with_clause as causing a library unit to become visible. However, that's not what the RM says. Instead, the RM says that *lack* of a with_clause causes a library unit *not* to be visible. That's kind of odd, but to understand the wording, you have to understand the existing model in this language-lawyerly fashion. The current RM defines the notion of an "environment declarative_part". During compilation, this fanciful declarative_part is imagined to contain all the library_items of interest. The order of these library_items is such that there are no forward semantic dependences. With_clauses introduce semantic dependences, so they control the order. Semantic dependences are also caused by child-parent and body-spec relationships. Subunits are imagined to be "inlined" at their stub point, so subunits can be ignored for visibility purposes. The visibility rules for library units come from treating the environment declarative_part in the usual way, with one difference: In a normal declarative_part, you can see everything that comes before, and nothing that comes after (with a bunch of hiding rules thrown in as well). In the environment declarative_part, however, you can see things that come before, but only if mentioned in a with_clause. Similar oddities occur in the definition of "immediate scope" and "scope". The best way to introduce limited_with_clauses into this model is to say that there is an implicit declaration of a "limited view of package X" (occurring before any "limited with X"'s). The normal declaration of X then appears later in its usual place. "Limited with X" introduces a very limited view of X that only includes nested packages and incomplete types. We need to define the nesting of with_clauses relative to each other and to use_clauses to phrase a number of rules that describe the interaction between use_clauses and nonlimited_ and limited_with_clauses appearing on different units. The changes to 10.1.2(8) ensure that, if for instance the specification of Q has a "with P.R", it shall not have a "limited with P" or a "limited with P.R". The same applies to the children of Q. On the other hand, a "limited with P.R.S" is fine. The fact that a "limited with P" hides the full view of P from all visibility prevents ripple effects. If you are is the scope of a "limited with P", the full view of P is hidden from all visibility, even if P would otherwise be visible indirectly (e.g. by a renaming in an auxiliary package). So the limited_with_clause effectively reverts to the limited view of P, and the types it declares are incomplete, and therefore subject to the limitations described in 3.10.1. From an implementation standpoint, it means that compilers can decide early (i.e., after looking at the context_clauses of a unit and of its ancestry) what view of P needs to be made available for name resolution purposes. It is still the case that the compiler may need to look at the full view of P, e.g. if some unit in the closure was compiled against the full view of P. Note that package instantiations and package renamings are not part of the limited view of a package, because we want to be able to build the limited view using very simple syntactic analysis (we are lucky that it's possible to identify tagged types simply by looking at the syntax). But for package renamings and instantiations, the name of the renamed package or of the generic would need to be resolved, and that would required full-fledged visibility analysis. To simplify implementation and avoid funky Beaujolais effects, use clauses are not allowed to mention a limited view of a package. Consider the example below; if the use clauses were legal, changing the "with A" to "with B" would silently change the meaning of X in P.C: package A is X : Integer := 42; end A; package B is X : Integer := 666; end B; limited with A; limited with B; use A, B; -- Illegal, really. package P is end P; with A; -- Change this to "with B" and it changes the meaning of X. package P.C is Y : Integer := X; end P.C; Furthermore, use clauses are inherited from the parent unit. This may be used to construct a situation where a limited_with_clause occurs in the scope of a use_clause for the same unit. For example: with A; package P is package R renames A; end P; with P; package Q is use P.R; -- Makes A use-visible. end Q; limited with A; package Q.C is -- A _not_ use-visible here. end Q.C; In this situation we say that the declarations in A are _not_ use-visible in the scope of the limited_with_clause. They are still potentially use-visible to prevent a Beaujolais effect illustrated by the following example (assume the same declaration for A and B as above): with A, B; package P is package RA renames A; package RB renames B; end P; with P; package Q is use P.RA, P.RB; -- Makes A and B use-visible. end Q; limited with A; -- Make this one a nonlimited "with"... with B; -- ... and this one a "limited with" package Q.C is Y : Integer := X; -- Illegal, really. end Q.C; The rules are such that both Xs are made potentially use-visible, so the name X is illegal. If P.RA.X was not made potentially use-visible, the name X would resolve to P.RB.X, and its meaning would change when switching the with_clause as explained in the example. If you nest (in that order) (1) a use_clause, (2) a limited_with_clause and (3) a nonlimited_with_clause, then the limited_with_clause effectively cancels the use_clause, but the nonlimited_with_clause doesn't reestablish it. Again, that's to prevent Beaujolais effects. A limited view of a package cannot be mentioned in a package renaming declaration. This restriction is probably not as critical as the others. The problem comes when someone from outside the unit containing the limited with references this package renaming. What does it see? If it has a "regular" with for the target of the limited-with, does it see the "full" view of the package via the renaming, or only the limited view of it. These are question that are best left unanswered. We do not allow a limited_with_clause to mention a procedure, because that would allow calling a procedure before its spec is elaborated. The language has no mechanism to deal with that. Similarly, we do not allow a limited_with_clause to mention a generic, because that would allow instantiating a generic before its spec is elaborated. We do not allow a limited_with_clause to mention a generic instantiation or a renaming, because that would require all kinds of semantic analysis stuff, such as visibility. The applicable nonlimited_ and limited_with_clauses determine (1) which entities are visible, (2) which dereferencing are legal and (3) what is the type of dereferences. In particular: 1 - Consider an expanded name P.X. X is visible if either the name is within the scope of a "with P" or if it is within the scope of a "limited with P" and it is part of the limited view (if other words, it is a nested type or package). 2 - Consider a dereference X.all, where X is of an access-to-incomplete type with designated type T. This dereference is illegal unless it occurs in the scope of a "with P" (where P is the package that declares T), within the immediate scope of the completion of T, or in a context where the expected type is the completion of T. 3 - Dereferencing an access-to-incomplete type (when it is legal) yields a name whose nominal subtype is the completion of T. This makes it possible to write an expression like X.all.Comp (assuming that we have the visibility necessary to know that Comp exists). There are various places in Chapter 8 that refer to "the declaration of a library unit", presuming that there's only one. We need to change the wording. One option is to say that library packages now have two declarations: the limited view, plus the normal package_declaration. Another option is to say that "the declaration of a library unit" is the normal package_declaration, the limited view being "something else". The latter option seems simpler, as it only requires to tweak 8.3(20) and doesn't interact with other parts of the language (e.g. freezing rules, library unit pragmas) like the former option would do. The part about the "declarative region of the incomplete type" in the wording of 3.10.1(10) is intended to forbid a case like: package P is private type T; type Ref is access T; Ptr : Ref; end P; with P; package body P is Too_Early : Integer := Ptr.all'Size; -- Illegal. type T is ...; end P; Here the dereference is illegal because it occurs in the declarative region of the incomplete type T, and the nonlimited_with_clause doesn't make it legal. Note that a dereference that would occur after the completion of T would not be of an incomplete type, so 3.10.1(10) would not apply; consequently, such a dereference would be legal even though it would be in the declarative region of the incomplete type T. Presuming that AI 262 is adopted, limited_with_clauses also combine with private_with_clauses, with the following syntax: limited private with P; Such a clause give visibility on the private view of P, but only at the beginning of the private part of the package where it appears. (The wording sections of AI 217 and AI 262 will obviously need to be merged when building the Draft International Standard.) !example !ACATS test !appendix From: Pascal Leroy Sent: Monday, February 3, 2003 5:19 AM A while back, Dan wrote, as a side comment: "Well, the nicest solution for the user is just to allow specs to with each other, as other languages do." This got me thinking, as this is an approach that we have discussed briefly once or twice, but writers of library-based compilers (like me) started to yell and scream that it would be impossible to implement. Well, I am starting to think that all approaches considered so far are nearly impossible to implement, and furthermore are quite hard to use. So I started to investigate a solution where units are allowed to with each others. Here are my half-baked ideas: 1 - A new form of context clause is added: weak [private] with library_unit_name {, library_unit_name} Weak with clauses don't have to form an acyclic graph. 2 - A compilation unit is added to the environment in two phases. During the first phase we say that the unit is "superficially added"; during the second phase that it is "fully added" (better terminology welcome). How this is achieved is not specified by the language. 3 - To superficially add a unit to the environment, there is no requirement that any other unit already exists in the environment. (Not even its parent, heck, not even Standard!) 4 - When a unit is superficially added to the environment, the only names that can be referenced from outside the unit are those of packages (both the unit itself and nested packages) and of types. Moreover, every type is at this point considered to be an incomplete type, and subject to the restrictions mentioned in 3.10.1; the taggedness of the types is visible. (The alert reader may notice some similarity between a unit superficially entered into the environment and a package abstract.) 5 - To fully add a unit to the environment, all of the units upon which it depends semantically must already have been fully entered into the environment. Furthermore, all of the units that it names in a weak with clause must already have been superficially entered into the environment. The only names that can be referenced from such units are those of types and packages, and types are considered incomplete. 6 - To fully add a unit to the environment, this unit must be made of the same lexical elements (except for comments) as when it was superficially added to the environment. 7 - The celebrated textbook example found in the AI is then simply written as follows: weak with Departments; package Employees is type Dept_Ptr is access all Departments.Department'Class; type Employee is tagged private; procedure Assign_Employee(E : in out Employee; D : in out Departments.Department); ... function Current_Department(D : in Employee) return Dept_Ptr; end Employees; weak with Employees; package Departments is type Emp_Ptr is access all Employees.Employee'Class; type Department is tagged private; procedure Choose_Manager(D : in out Department; Manager : in out Employees.Employee); ... end Departments; 8 - Marketing hype a la Tucker: - No three-part types - No new compilation units - No new declarations - No changes to the visibility rules - Little new syntax - Support for circularities involving types in nested packages I am no expert in source-based compilers, but I believe that this model should be quite close to what GNAT currently does for "with type". Superficially adding a unit to the environment would merely mean that the source file exists, and the "weak with" would give the compiler permission to peak at it. For library-based compilers this proposal adds some complexity, but not necessarily more than those proposals which change the visibility rules or otherwise break the invariants upon which the compilers depend. In the case of our implementation a tree in the library can be in two states: parsed (as produced by the parser, no names resolved) or analyzed (with all the decorations produced by the semantic analyzer, names resolved, etc.). These states would correspond directly to a unit superficially/fully entered in the environment. We would need to beef up the parsed representation a bit to add (1) tables listing the types and package names, and (2) a hash code representing the sequence of tokens in the unit (to perform the check of #6 above). Intense hand-waving here, as I have not done anything resembling a serious design. Feel free to explain why this is a dumb idea... ************************************************************** From: Arnaud Charlet Sent: Monday, February 3, 2003 5:35 AM I certainly like Pascal's proposal (weak with) very much. It is the only proposal (apart from the old with type that had its bunch of issues) so far that looks appealing to me. It addresses well the issue of interfacing with other languages and from a user point of view, it is very close to an ideal solution. As Robert said, we don't have any real technical constraint in GNAT to implement any of the proposed solution. This one would probably be close to the current with type implementation. I am not sure the wording about adding a unit in the environment, and requiring compilers to have two (or more for that matter) phases should be stated as such, but I guess this can probably be sorted out without too much difficulty if this proposal gets some more supporters. ************************************************************** From: Robert Dewar Sent: Monday, February 3, 2003 6:39 AM Pascal's proposal for weak with is the first one I have seen since WITH TYPE that I like, and in fact I like it better than WITH TYPE (I am not taling deep semantics here, just a surface impression). It also seems to satisfy the requirement for easy generation of equivalent Ada from e.g. Java stuff. ************************************************************** From: Robert Dewar Sent: Monday, February 3, 2003 6:42 AM By the way I don't like "weak" very much because it has the wrong connotations to be. The issue is that this with is preliminary or limited (so if you are in a mood to reuse keywords. limited with x; is more like it ... A more radical proposal is that all WITH's that meet the criteria are weak -- could that work? ************************************************************** From: Pascal Leroy Sent: Monday, February 3, 2003 6:51 AM I am not wedded to a particular syntax. If we think that the idea is worth pursuing, we can probably come up with an acceptable syntax (we don't lack creativity in the area of syntax). > A more radical proposal is that all WITH's that meet the criteria are > weak -- could that work? From a methodological standpoint, I'd say that introducing a circularity should be the result of a conscious decision, not something that happens by chance. So the user should have to make it explicit that a circularity is what she wants--by typing "weak", "limited" or whatever. ************************************************************** From: Robert Dewar Sent: Monday, February 3, 2003 7:03 AM I am not so sure I agree with this. At one level of abstraction you WITH something because you need something in it, and it is not really your business if it needs something in yourself. ************************************************************** From: Jean-Pierre Rosen Sent: Monday, February 3, 2003 7:02 AM > limited with x; > > is more like it ... > Or even with X abstract; ;-) ************************************************************** From: Robert A. Duff Sent: Monday, February 3, 2003 8:18 AM > A while back, Dan wrote, as a side comment: "Well, the nicest solution for the > user is just to allow specs to with each other, as other languages > do." Dan was exactly correct. That's why I can't get too excited about Tucker's aesthetic gut feelings about the various other proposals -- they are *all* less aesthetically pleasing than the "right" solution. >...This > got me thinking, as this is an approach that we have discussed briefly once or > twice, but writers of library-based compilers (like me) started to yell and > scream that it would be impossible to implement. Well, I am starting to think > that all approaches considered so far are nearly impossible to >implement, ... That seems exaggerated, ... >...and > furthermore are quite hard to use. So I started to investigate a solution where > units are allowed to with each others. Here are my half-baked ideas: > > 1 - A new form of context clause is added: > > weak [private] with library_unit_name {, library_unit_name} > > Weak with clauses don't have to form an acyclic graph. This is the solution I chose for my from-scratch language design I do as a hobby. My syntax is "import X;" to import X and require X to be elaborated earlier than this unit -- essentially the same as Ada's "with X;" And "import forward X;" to import X and elaborate X *after* this unit. This is essentially the same as your "weak with X;". ("import forward" is like a forward reference.) The only thing needed in the RM is the syntax for weak with's, make sure the rule about no cycles applies only to cycles formed by strong with's. All the stuff below is really a description of how the Rational compiler would work (and others with a similar library model). There's no need for any of it in the RM. (Aside: In my "hobbyist" language, I am able to achieve complete *compile-time* checking of elaboration order problems, even in the case of indirect/dispatching calls, with no sacrifice in expressive power. I thought that was pretty cool.) > 2 - A compilation unit is added to the environment in two phases. During the > first phase we say that the unit is "superficially added"; during the second > phase that it is "fully added" (better terminology welcome). How this is > achieved is not specified by the language. > > 3 - To superficially add a unit to the environment, there is no requirement that > any other unit already exists in the environment. (Not even its parent, heck, > not even Standard!) > > 4 - When a unit is superficially added to the environment, the only names that > can be referenced from outside the unit are those of packages (both the unit > itself and nested packages) and of types. Moreover, every type is at this point > considered to be an incomplete type, and subject to the restrictions mentioned > in 3.10.1; the taggedness of the types is visible. (The alert reader may notice > some similarity between a unit superficially entered into the environment and a > package abstract.) > > 5 - To fully add a unit to the environment, all of the units upon which it > depends semantically must already have been fully entered into the environment. > Furthermore, all of the units that it names in a weak with clause must already > have been superficially entered into the environment. The only names that can > be referenced from such units are those of types and packages, and types are > considered incomplete. > > 6 - To fully add a unit to the environment, this unit must be made of the same > lexical elements (except for comments) as when it was superficially added to the > environment. Note in particular that this consistency check is an artifact of your compilation model. That is, your compiler is going to read the same file twice. But I can imagine compilers that will only read the source once, so the "superficial" and "full" distinction is represented purely in memory during compilation. I think any consistency requirements are already covered by the RM's words about not including two different versions of the same unit in a partition. > 7 - The celebrated textbook example found in the AI is then simply written as > follows: > > weak with Departments; > package Employees is > type Dept_Ptr is access all Departments.Department'Class; > type Employee is tagged private; > procedure Assign_Employee(E : in out Employee; > D : in out Departments.Department); > ... > function Current_Department(D : in Employee) return Dept_Ptr; > end Employees; > > weak with Employees; > package Departments is > type Emp_Ptr is access all Employees.Employee'Class; > type Department is tagged private; > procedure Choose_Manager(D : in out Department; > Manager : in out Employees.Employee); > ... > end Departments; Yes, although only one of the with's *needs* to be weak. For symmetry, they probably both should be weak. > 8 - Marketing hype a la Tucker: > > - No three-part types > - No new compilation units > - No new declarations > - No changes to the visibility rules > - Little new syntax > - Support for circularities involving types in nested packages > > I am no expert in source-based compilers, but I believe that this model should > be quite close to what GNAT currently does for "with type". Superficially > adding a unit to the environment would merely mean that the source file exists, > and the "weak with" would give the compiler permission to peak at it. > > For library-based compilers this proposal adds some complexity, but not > necessarily more than those proposals which change the visibility rules or > otherwise break the invariants upon which the compilers depend. In the case of > our implementation a tree in the library can be in two states: parsed (as > produced by the parser, no names resolved) or analyzed (with all the decorations > produced by the semantic analyzer, names resolved, etc.). These states would > correspond directly to a unit superficially/fully entered in the environment. > We would need to beef up the parsed representation a bit to add (1) tables > listing the types and package names, and (2) a hash code representing the > sequence of tokens in the unit (to perform the check of #6 above). Intense > hand-waving here, as I have not done anything resembling a serious design. > > Feel free to explain why this is a dumb idea... Because your evil twin was yelling and screaming about the implementation difficulty of this idea... ;-) ************************************************************** From: Tucker Taft Sent: Monday, February 3, 2003 8:24 AM I like this, although it is more work for us than almost any other proposal. We do all semantics on the fly, and don't have a mode that parses without doing semantic analysis. In general, I believe the "type C.T;" approach solves the basic problem, and will be one of the easiest to implement. As far as syntax, I like the suggestion of using "limited" as a prefix on the "with" clause, and it goes awfully nicely with "private" ;-). [limited] [private] with library_item_name {, library_item_name}; It would be a major headache (dare I say "huge" headache?) for us if the "limited"ness of the "with" were not explicit. ************************************************************** From: Robert A. Duff Sent: Monday, February 3, 2003 8:34 AM Robert wrote: > A more radical proposal is that all WITH's that meet the criteria are > weak -- could that work? No, I don't think so. The problem is that if there's a cycle of with's, the compiler needs to choose *one* of them as "weak" to break the cycle. The one that is chosen affects the elaboration order. Clearly, the user needs a way to control this order, and "weak with" is that feature. In other words, how is the compiler to know *which* with's meet the criteria? I don't think the usual army of elaboration-control pragmas can help. Pragma Elaborate(X) should probably be illegal for weak withs. Anyway, it only goes one step, which as we know from Ada 83 is not good enough. Pragma Elaborate_All should ignore weak withs that are found in the chain. Otherwise, it can't work when used in part of the cycle. Lots of other languages do as Robert suggests -- just allow cyclic imports, with no indication of which ones are "weak". The reason they can get away with it, and we can't is: Some such languages don't have run-time elaboration semantics, so there's no issue. Other such languages have botched the run-time elaboration semantics so badly that this issue is in the noise. > I am not so sure I agree with this. At one level of abstraction you WITH > something because you need something in it, and it is not really your > business if it needs something in yourself. I agree with Pascal on this point. It seems to me that a mutually recursive design always requires *some* mutually recursive knowledge on the part of the designer. I don't see it happening by accident. At least not unless the packages involved are incoherent collections of unrelated stuff. Normal layered "with"s have the property that packages don't need to know about their clients. Mutual recursion seems rather different to me: two packages need to know about each other -- they at least need to know about each other's *existence*. So I think weak with's deserve their own syntax. I guess I agree with Robert that "weak with" is not the best syntax, but that's a minor detail. Using "with" as a verb is already an annoyance... Oh, well. ************************************************************** From: Robert Dewar Sent: Monday, February 3, 2003 8:43 AM > No, I don't think so. The problem is that if there's a cycle of with's, > the compiler needs to choose *one* of them as "weak" to break the > cycle. The one that is chosen affects the elaboration order. Clearly, > the user needs a way to control this order, and "weak with" is that > feature. > > In other words, how is the compiler to know *which* with's meet the > criteria? Yes, that makes sense, so that's not a possibility that is worth discussing. I withdraw the solution. ************************************************************** From: Pascal Leroy Sent: Monday, February 3, 2003 8:37 AM > The only thing needed in the RM is the syntax for weak with's, make sure > the rule about no cycles applies only to cycles formed by strong with's. > > All the stuff below is really a description of how the Rational compiler > would work (and others with a similar library model). There's no need > for any of it in the RM. That statement surprises me. At the very least, it would seem that the RM would need to specify what names/entities are available after a unit has been superficially entered into the environment, and what are the properties of these entities. I am proposing that you can see only packages and types, and even so the types would look like they are incomplete. I can't see how you could deduce this from the current RM. PS: But then I know that Bob Duff, like the Pope, is infallible in matters of language doctrine ;-) ************************************************************** From: Robert A. Duff Sent: Monday, February 3, 2003 8:51 AM Yes, you're right -- the RM needs to say *something*. But it's not much. And I don't think it needs to define this concept of "superficially in the environment" -- that concept belongs in the Rational user's manual, I think. The RM should say something about how if the only visibility to package X is via "weak with", then uses of X are restricted along the lines you say above. More thought is needed, if we are to go this route. The only real problem I see is implementation difficulty. I think all the language rules can be worked out, but that will require some thought. Interactions with elaboration-control pragmas. What happens when you've got both a weak and a strong with on the same thing? Does "weak with A.B.C;" import all three weakly? Is there any concern that changing "with" to "weak with", or vice versa, would introduce surprises into the program? I suspect all these questions have easy straightforward answers. > PS: But then I know that Bob Duff, like the Pope, is infallible in matters of > language doctrine ;-) ;-) No way! ************************************************************** From: Robert A. Duff Sent: Monday, February 3, 2003 12:56 PM In Pascal's proposal, a parent package P can weakly 'with' its children: "weak with P.C;", if you want a mutually dependent pair of types, one in P and one in P.C. The spec of P will be elaborated before the spec of P.C, as usual. However, it is pointless for P.C to say "weak with P;", because the child-to-parent dependency is necessarily strong (given that there's no proposal to add syntax saying otherwise -- "weak children?"). No problems here. I'm just making some observations. I guess I favor Robert's proposed "limited with" syntax. ************************************************************** From: Robert Eachus Sent: Monday, February 3, 2003 2:26 PM So do I, and I am beginning to like this proposal. As far as I can see it is clearly better than package abstracts in two ways, there is no new compilation unit type, and there is no problem of keeping the abstract consistant with the package itself. In other words from a theoretical point of view, the "limited with" creates a package abstract and then withs it. Since the completion of the types and the subprograms (and parameters) get lost in the process, there is no visible circularity. But I think that there are two "gotchas" that have to be dealt with. The first is where you have a type declaration that depends on another type: type Color is (Red, Green, Blue); type Properties(C: Color) is...; Does the incomplete type for Properties include the discriminant? Choices are yes, and it is illegal to do anything with it, type Color is not incomplete, and the type Properties has undefined discriminants. (Others?) I prefer the choice where only private, record and tagged types are incomplete in the limited view. This brings me directly to the second gotcha. I think it is clear (at least to me) that for this to be useful, the limited view of an access type has to be an access type. But there have to be limits on how this access type can be used in the limited view. (Translation, if you create a type with a component of (a limited view of) an access type, then it is illegal to have a non-null default value for a component of that type. Other rules are possible.) ************************************************************** From: Robert A. Duff Sent: Monday, February 3, 2003 2:26 PM > This brings me directly to the second gotcha. I think it is clear (at > least to me) that for this to be useful, the limited view of an access > type has to be an access type. But there have to be limits on how this > access type can be used in the limited view. Can't we say that it's an access type whose designated type is incomplete? And then you can't do things like X.all. >... (Translation, if you > create a type with a component of (a limited view of) an access type, > then it is illegal to have a non-null default value for a component of > that type. Other rules are possible.) I think you can't have an allocator (as a default or anything else), but I don't see why it couldn't be a function call that happens to return the access type. Perhaps an example? ************************************************************** From: Dan Eilers Sent: Monday, February 3, 2003 3:04 PM I like Pascal's "weak with" proposal a lot, although I'm not yet convinced we need to syntactically distinguish "weak with" from normal withs. I came to this conclusion by perhaps a slightly different route: 1) we have a bunch of proposed syntax extensions to solve the circular types problem, any of which are believed to be feasible to implement; 2) essentially none of the proposed syntax extensions have any dual-use benefit towards the solution of any other problem; 3) we envision a tool that can automatically translate circular specs written in other languages into the proposed new syntax; 4) a similar tool could also translate Ada that allowed circular with's into any of the proposed syntax extensions; 5) given such a tool, the users would happily write Ada with circular with's, and use the tool to create their package abstracts (in whatever syntax); 6) this could be made automatic. When the compiler sees a "with" clause for a unit that has not been compiled, instead of giving an error, it can automatically invoke the translation tool to create the package abstract, and then compile the abstract before continuing to compile the original unit. 6) this can be further optimized if desired so the translation tool is integrated into the compiler, rather than needing to be a stand-alone tool. Notes: Recursive subprogram calls don't use different syntax than non-recursive ones, so ideally recursive with's wouldn't either. We need to address the elaboration order concerns, but presumably that is more naturally done with elaboration pragmas than with different syntax for "with" clauses. I still think there is some possible dual-use synergy with the separate private issue. As Tuck corrected me, you can't just put the private part in a separate file. So instead of "private is separate" you would have to say "the remainder of this package is separate". A "with" clause on such a package would continue to see both the root and the subunit. A "with p'abstract" would see only the root. I believe such new syntax would at the same time solve the separate private issue and the circular types issue, and perhaps also the separate constants issue. ************************************************************** From: Robert Dewar Sent: Monday, February 3, 2003 4:07 PM > 6) this could be made automatic. When the compiler sees a "with" clause > for a unit that has not been compiled, instead of giving an error, it > can automatically invoke the translation tool to create the package > abstract, and then compile the abstract before continuing to compile > the original unit. That won't do at all for a source based compilation system, there is no concept of "a unit that has not been compiled". One of the critical guarantees in GNAt is that order of compilation NEVER affects the results and we would strenuously object to a change in semantics which would destroy this guarantee. For one thing it means that the meaning of a program is not fully conveyed by its sources, and that is pragmatically and philsophically unacceptable. ************************************************************** From: Robert A. Duff Sent: Monday, February 3, 2003 4:16 PM Yes, I agree. This makes me think that these tools that translate from other languages (or create bindings thereto, I guess) ought to make sure that their generated Ada code doesn't do anything at elab time -- at least in the presence of cycles. That seems achievable, and simpler than having user options to specify elab order. ************************************************************** From: Robert Dewar Sent: Monday, February 3, 2003 4:25 PM Note that another objection is that this means that automated tools for determining the compilation order would have to do semantic analysis. ************************************************************** From: Robert A. Duff Sent: Monday, February 3, 2003 4:12 PM Dan Eilers wrote: >... > 3) we envision a tool that can automatically translate circular specs > written in other languages into the proposed new syntax; Right, but those languages typically don't have the same elaboration-order issues that Ada does. > 4) a similar tool could also translate Ada that allowed circular with's > into any of the proposed syntax extensions; True, but... > 5) given such a tool, the users would happily write Ada with circular > with's, and use the tool to create their package abstracts (in > whatever syntax); I fear that such a tool would be inherently error prone. Suppose we have package specs A and B, which 'with' each other. The tool would have to choose which one to elaborate first, and it might guess wrong. I think it's better to let the user specify the order via "limited with". Besides, now that I think about it, how would this tool work? "Limited with" implies a very restricted usage of the imported package. The tool would have to check that such restrictions are obeyed before turning a "with" into a "limited with". > 6) this could be made automatic. When the compiler sees a "with" clause > for a unit that has not been compiled, instead of giving an error, it > can automatically invoke the translation tool to create the package > abstract, and then compile the abstract before continuing to compile > the original unit. Users normally don't compile. They run a build tool, which automatically compiles everything that needs to be compiled, in some order that is not easily known to the programmer. It seems error-prone to have the semantics depending on what happens to have already been compiled. That's true even if the user bypasses the build tool and explicitly invokes the compiler in some order. > 6) this can be further optimized if desired so the translation tool is > integrated into the compiler, rather than needing to be a stand-alone > tool. > > > Notes: > > Recursive subprogram calls don't use different syntax than non-recursive > ones, so ideally recursive with's wouldn't either. True, but this case seems different to me. If I see: with B; package A is ... I currently know for certain that the spec of B is elaborated before A. I don't have to go looking at B to see that that's true. I don't think we should break that property. > We need to address the elaboration order concerns, but presumably that > is more naturally done with elaboration pragmas than with different > syntax for "with" clauses. I don't think the existing pragmas can be used for this purpose. Suppose the cycle is "A with's B with's C with's A". Adding a pragma Elaborate in A pointing to B doesn't work, because it's not transitive to C. Adding Elaborate_All doesn't work, because that would require A to be elaborated before itself (because it's transitive). So we would have to add a new pragma to explicitly break one link in the cycle: with X; pragma Don't_Elaborate(X); -- ;-) But that means precisely what "limited with X;" would mean. Elaboration pragmas were a mistake in the first place. They were added late in the game to Ada 83 to solve the "Brosgol anomalies". We added more pragmas in Ada 95, and I'm afraid we botched the job -- we *still* didn't solve the problems. I suggest we avoid compounding the mistake of using pragmas to control elaboration. Furthermore, "limited with X" implies various restrictions on the use of X -- you can only refer to types in X, and they are all incomplete (or access to incomplete). Pragmas shouldn't be in *that* business. > I still think there is some possible dual-use synergy with the separate > private issue. As Tuck corrected me, you can't just put the private > part in a separate file. So instead of "private is separate" you would > have to say "the remainder of this package is separate". A "with" clause > on such a package would continue to see both the root and the subunit. > A "with p'abstract" would see only the root. I believe such new syntax > would at the same time solve the separate private issue and the circular > types issue, and perhaps also the separate constants issue. Perhaps, but I skept. I'm having trouble seeing the separate-private issue as anything more than a simple textual translation, with no run-time consequences whatsoever. I haven't seen a fully worked-out proposal that solves both problems... P.S. This discussion makes me think of another observation. If a given cycle contains two or more "limited with"s, then elaboration order will presumably be implementation defined. I suppose that's OK -- if you care, you can break the cycle with just one "limited with", and use normal "with" for the rest. ************************************************************** From: Dan Eilers Sent: Monday, February 3, 2003 6:23 PM Bob Duff wrote: > Perhaps, but I skept. I'm having trouble seeing the separate-private > issue as anything more than a simple textual translation, with no > run-time consequences whatsoever. I haven't seen a fully worked-out > proposal that solves both problems... OK, here's the canonical example worked out: -- Note: with'ing package P creates a semantic dependency on all subunits -- such as P.Extension declared in P, and makes all declarations in -- P.Extension visible as if they were declared directly in P. -- with'ing P'abstract does not create a semantic dependency on subunits -- such as P.Extension, and does not make p.Extension's declarations visible, -- and does not allow declaring objects of types from P whose full type is -- deferred to P.Extension. package Employees is type Employee is private; -- completed in subunit type Emp_Ptr is access all Employee; package Extension is separate; end Employees; package Departments is type Department is private; type Dept_Ptr is access all Department; package Extension is separate; end Departments; with Departments'abstract; separate(Employees) package Extension is type Emp_Ptr is access all Employee; procedure Assign_Employee(E : access Employee; D : access Departments.Department); function Current_Department(D : access constant Employee) return Departments.Dept_Ptr; private type Employee is record dept: Departments.Dept_Ptr; end record; end Extension; with Employees'abstract; separate(Departments) package Extension is type Dept_Ptr is access all Department; procedure Choose_Manager(D : access Department; Manager : access Employees.Employee); private type Department is mgr: Employees.Emp_Ptr; end record; end Extension; For the "private is separate" problem, instead of: package p is ... private is separate; end P; you would have: package p is ... private package extension is separate; end; ************************************************************** From: Robert A Duff Sent: Tuesday, February 11, 2003 6:07 PM Dan Eilers reminded me in private e-mail to look at this e-mail he sent a week ago. It does seem like a reasonable proposal, and it does seem like it allows the full type for a private type to go in a separate file. I presume the rules for completing private types would be relaxed to allow the full type in a package subunit. I presume that if you say "with P'abstract", then P.T is an incomplete type (view). This proposal seems very similar to the one Tucker proposed a while ago, where package P declares that it has a child C. The syntax here is more "subunit like" rather than "child like". Dan wrote: > Bob Duff wrote: > > > > I still think there is some possible dual-use synergy with the separate > > > private issue. As Tuck corrected me, you can't just put the private > > > part in a separate file. So instead of "private is separate" you would > > > have to say "the remainder of this package is separate". A "with" clause > > > on such a package would continue to see both the root and the subunit. > > > A "with p'abstract" would see only the root. I believe such new syntax > > > would at the same time solve the separate private issue and the circular > > > types issue, and perhaps also the separate constants issue. > > > > Perhaps, but I skept. I'm having trouble seeing the separate-private > > issue as anything more than a simple textual translation, with no > > run-time consequences whatsoever. I haven't seen a fully worked-out > > proposal that solves both problems... > > OK, here's the canonical example worked out: > > -- Note: with'ing package P creates a semantic dependency on all subunits > -- such as P.Extension declared in P, and makes all declarations in > -- P.Extension visible as if they were declared directly in P. > > -- with'ing P'abstract does not create a semantic dependency on subunits > -- such as P.Extension, and does not make p.Extension's declarations visible, > -- and does not allow declaring objects of types from P whose full type is > -- deferred to P.Extension. By "not allow declaring objects" I presume all the limitations of incomplete types are applied. You can declare access-to-it, and you can declare parameters (if it's tagged), I guess. > package Employees is > type Employee is private; -- completed in subunit > type Emp_Ptr is access all Employee; > package Extension is separate; > end Employees; > > package Departments is > type Department is private; > type Dept_Ptr is access all Department; > package Extension is separate; > end Departments; > > with Departments'abstract; > separate(Employees) > package Extension is > type Emp_Ptr is access all Employee; I don't see why you want to declare Emp_Ptr here. > procedure Assign_Employee(E : access Employee; > D : access Departments.Department); > function Current_Department(D : access constant Employee) return > Departments.Dept_Ptr; > private > type Employee is record > dept: Departments.Dept_Ptr; > end record; > end Extension; > > with Employees'abstract; > separate(Departments) > package Extension is > type Dept_Ptr is access all Department; > procedure Choose_Manager(D : access Department; > Manager : access Employees.Employee); > private > type Department is > mgr: Employees.Emp_Ptr; > end record; > end Extension; Hmm. The existing language allows: package P is type T is private; package Q is private type T is new Integer; end Q; private type T is new Boolean; end P; And the type Q.T has nothing to do with P.T. I'm not sure how the rules would deal with this sort of thing. Exactly when is a full type considered to complete a private type, and how do we prevent duplicate completions. The current language always completes things in the same declarative region, so this would add the complexity of completing things elsewhere. Well, now that I think about it, I take that back, somewhat: An accept_statement can be more nested than the corresponding entry, and there can be more than one of them. So that's a similar situation. > For the "private is separate" problem, instead of: > > package p is > ... > private is separate; > end P; > > you would have: > > package p is > ... > private > package extension is separate; > end; Seems plausible, although I'm still not sure we need to combine the solutions to these two problems. After all, the separate-private as a purely textual translation seems OK. Along with the "private with" idea. ************************************************************** From: Dan Eilers Sent: Tuesday, February 11, 2003 7:40 PM Bob Duff wrote: > It does seem like a reasonable proposal, and it does seem like it allows > the full type for a private type to go in a separate file. ... > This proposal seems very similar to the one Tucker proposed a while ago, > where package P declares that it has a child C. The syntax here is more > "subunit like" rather than "child like". Yes, the two proposals are very similar. They both involve an incomplete type whose completion is deferred to a child or stub that is mentioned in the parent. The advantages to using subunits are: 1) the subunit proposal leaves the clients unchanged. The child proposal requires library-level renames in order to leave the clients unchanged; 2) it is natural in Ada for a parent to declare a subunit, but not for a parent to declare a child unit; 3) the subunit proposal serves a dual purpose in solving the need for separately compiled private parts; The advantage to using child units is: 1) the subunit proposal requires new syntax for "with" clauses, to indicate no semantic dependence on any subunits. > Seems plausible, although I'm still not sure we need to combine the > solutions to these two problems. After all, the separate-private as a > purely textual translation seems OK. Along with the "private with" > idea. This "purely textual translation" is essentially "pragma include" from pre-Ada83 days. It was excluded from Ada in favor of subunits because: 1) subunits often need their own context clauses, which include files don't provide; 2) subunits specify their parent and can only have one parent; 3) subunits are well-formed, and can be compiled independently from their parent These considerations also apply to separately compiled private parts, making subunits a much better solution than the combination of two hacks: "pragma include" and "private with". ************************************************************** From: Tucker Taft Sent: Monday, February 3, 2003 4:27 PM Robert A Duff wrote: > > > This brings me directly to the second gotcha. I think it is clear (at > > least to me) that for this to be useful, the limited view of an access > > type has to be an access type. But there have to be limits on how this > > access type can be used in the limited view. > > Can't we say that it's an access type whose designated type is > incomplete? And then you can't do things like X.all. You are starting to give me the "willies." We will have to process rep-clauses if the access type in a limited-with-ed package is not itself considered an incomplete type. If we want a non-incomplete access type, then I think we need to declare it in a non-limited-withed place. Hence, this doesn't solve the problem associated with proliferation of access type declarations. And I do still like that "type C.T;" ;-). (what a pain I can be...) > >... (Translation, if you > > create a type with a component of (a limited view of) an access type, > > then it is illegal to have a non-null default value for a component of > > that type. Other rules are possible.) > > I think you can't have an allocator (as a default or anything else), > but I don't see why it couldn't be a function call that happens to > return the access type. Perhaps an example? Gack. You are back to requiring that the limited-with-er chooses a representation for a type that it isn't declaring. In this era where we are on the cusp of moving from 32-bit pointers to 64-bit pointers, I fear that rep clauses on access types may be more common, rather than less. By the way, I have discovered (remembered?) that we have a "loose parser" built into our compiler. It's job is to do lookahead where strict one-pass semantic analysis is difficult. For example, it scans bodies looking for labels, and then inserts them at the "begin." It scans generic formal parts looking ahead until it finds out in what parent unit the generic is declared, so it can interpret the names in the formal part properly. In any case, this is a full, but very stupid and quite "lax" parser. It could probably build up a tree of package and type names. It could certainly not make any semantic sense out of a rep clause, and since the rep clause could use named numbers declared in other packages (e.g. "for My_Acc_Type'Size use Targ_Dep.Size_Of_Access_Type;"), it really couldn't be made to work. So if we consider this limited-with approach, I think we want to make minimal assumptions about syntactic processing, and zero assumptions about representation processing, of the items in the limited-withed unit. I also agree with Bob that the less said about the way the magic happens the better. Clearly there will be some kind of compilation dependence on the source of the limited-withed unit, but the key point is there is no semantic or elaboration dependence on the unit. It seems fine if the source of the unit is edited "after" something that mentioned it in a limited-with clause is compiled, but before the unit itself is compiled. But of course, whenever the source is changed, everything that has a compilation-dependence on it will probably need to be recompiled prior to linking. The "trick" that makes it all possible is that the *fully compiled* representation of both units in a cyclic dependence depend on the *source text* of both units. The fully compiled representations don't both depend on the others fully compiled representation. Of course, there may be intermediate forms between source text and fully compiled representations, but those don't need to be mentioned. And the other important point is there is no transitive compilation dependence if the unit mentioned in a limited-with clause itself depends on other units, via withs or limited-withs. Presumably, mutual inlining, if supported, requires similar tricks. Other observations if we consider limited-with: I suppose to be consistent, we should interpret "limited with P.Q.R;" as equivalent to "limited with P, P.Q, P.Q.R;". Also it seems clear that limited with is ignored if there is a non-limited with for the same unit. Another important point is that if a unit is mentioned in a limited with clause, it is a *post*-compilation rule that the unit be semantically (or even syntactically) correct, as opposed to a legality rule. When compiling the unit having the limited with, the compiler may of course check some part of this post-compilation rule early (provided of course it doesn't create an inappropriate semantic dependence), but an ACATS test should not require that any particular errors in the unit mentioned in the limited-with be identified at this time. Certainly compilers will vary in how picky is the parser that they use to implement limited with, especially given that Ada's "syntax" rules slop over into semantic processing fairly frequently, due to the lack of direct LALR(1) parsability. The only requirement should be that *if* the unit is semantically correct, then you can safely mention it in a limited-with. But you cannot rely on the compiler detecting all (or any) errors in a unit mentioned in a limited-with, though it can be arbitrarily picky about it. Eventually, prior to linking, it will require that the unit exist and be semantically correct. This means that when a unit mentions a second unit in a limited-with clause, the second unit is "needed by" the first unit, in the sense of 10.2(2-6). The above is related to 10.1.4(5) which says: When a compilation unit is compiled, all compilation units upon which it depends semantically shall already exist in the environment... We could change this to "... upon which it depends semantically or which it mentions in a limited_with_clause shall ..." and it would still be OK. However, 10.1.4(6) which says: The implementation may require that a compilation unit be legal before inserting it into the environment. is too strict. It could be changed to: The implementation may require that a compilation unit satisfy the Syntax Rules before inserting it into the environment. The implementation may require that a compilation unit be legal before allowing it to be mentioned in a with_clause other than a limited_with_clause. ************************************************************** From: Robert A Duff Sent: Monday, February 3, 2003 5:17 PM Tucker wrote: > Robert A Duff wrote: > > Can't we say that it's an access type whose designated type is > > incomplete? And then you can't do things like X.all. > > You are starting to give me the "willies." > [...description of willies] OK, I withdraw my suggestion. But it seems like whatever solution to the mutual recursion problem we choose, we really need to solve the problem of proliferation of silly type conversions. Otherwise, these proposals will be nearly unusable. That is, there ought to be an *implicit* conversion between access types in the "safe" cases. I know we discussed that before. Is it part of any of these proposals, or is it a separate AI? I seem to recall that it might be wrapped up with other access-type issues, like allowing "not null" constraints, and access-to-constant parameters. > And I do still like that "type C.T;" ;-). > (what a pain I can be...) That's a fine solution, IMHO. Your earlier e-mail about mutually-recursive types being part of the same abstraction makes me comfortable with the idea of tying the solution to child packages. (See, it *is* possible to convey sensible information about purely aesthetic concerns. ;-)) But I would choose the "limited with" idea, unless it is considered too hard to implement. > Presumably, mutual inlining, if supported, requires similar tricks. And mutual generic instantiation? > Other observations if we consider limited-with: > > I suppose to be consistent, we should interpret "limited with P.Q.R;" > as equivalent to "limited with P, P.Q, P.Q.R;". Also it seems clear > that limited with is ignored if there is a non-limited with for the > same unit. Or any other "strong" dependence, such as child upon parent. I guess "strong dependence", as I've been calling it, is exactly the same notion as "semantic dependence". > Another important point is that if a unit is mentioned in a limited with > clause, it is a *post*-compilation rule that the unit be semantically > (or even syntactically) correct, as opposed to a legality rule. ... I don't see the need for any verbiage in the RM about this point. And I don't see how this differs (in theory) from normal with_clauses. (It might differ in practise, depending on how the compiler actually works.) I mean, if B says "with A;", there is currently no rule saying "A shall be legal." A compiler could properly say nothing more than "no errors found in B", even if there are errors in A. (That would be unfriendly, IMHO.) Of course, sometime before running the program, A must be compiled, and the errors detected. > ...This > means that when a unit mentions a second unit in a limited-with > clause, the second unit is "needed by" the first unit, in the > sense of 10.2(2-6). Yes, that's a key point. It might be that the second unit is mentioned *only* in limited_with_clauses, in which case there are no elaboration dependendences upon it, but it still needs to be included in the program, and elaborated at some point (possibly very late). > The above is related to 10.1.4(5) which says: > > When a compilation unit is compiled, all compilation units upon which > it depends semantically shall already exist in the environment... > > We could change this to "... upon which it depends semantically or > which it mentions in a limited_with_clause shall ..." and it would still be OK. I don't see any need to change this, given that "existing in the env" is such a vague concept. In GNAT, it just means the file is sitting there on the disk. In AdaMagic, it just means it is sitting there on the disk and has been "registered". Registration only does minimal syntax checking. In Rational's compiler, it means something more. > However, 10.1.4(6) which says: > > The implementation may require that a compilation unit be legal before > inserting it into the environment. > > is too strict. It could be changed to: I don't see why. The GNAT model does not take advantage of this permission. The permission is there for the benefit of compilers like Rational, where "inserting" involves running the compiler, and therefore checking various rules. I suppose AdaMagic takes partial advantage of this permission, since insertion involves some small amount of syntax checking. But why should we care how much checking is done on B before "limited with B" can be swallowed by the compiler? If B is illegal, the compiler can process "limited with B" if it likes, or (if it happens to notice the illegality), it can complain that "limited with B" is nonsense. It doesn't matter whether the illegality is syntactic or semantic. > The implementation may require that a compilation unit satisfy the Syntax > Rules before inserting it into the environment. The implementation may > require that a compilation unit be legal before allowing it to be mentioned > in a with_clause other than a limited_with_clause. I see no need for this added complexity. And it could be a burden -- if the compiler chooses to implement some simple Legality Rule in the parser, that should be allowed. ************************************************************** From: Tucker Taft Sent: Monday, February 3, 2003 6:20 PM Robert A Duff wrote: > Tucker wrote: > ... >>The above is related to 10.1.4(5) which says: >> >> When a compilation unit is compiled, all compilation units upon which >> it depends semantically shall already exist in the environment... >> >>We could change this to "... upon which it depends semantically or >>which it mentions in a limited_with_clause shall ..." and it would still be OK. > > I don't see any need to change this, given that "existing in the env" is > such a vague concept. In GNAT, it just means the file is sitting there > on the disk. In AdaMagic, it just means it is sitting there on the disk > and has been "registered". Registration only does minimal syntax > checking. In Rational's compiler, it means something more. I think we need to say *something* about the requirements on a unit mentioned in a limited-with clause. Why not say it must exist in the environment? As you said, that is pretty vague. If there are no requirements on it, then the implementation has no place to "hang" whatever requirements it really does have. It can't just invent requirements that don't exist. But it can say, "as far as our implementation is concerned, that unit doesn't exist in the environment, so we refuse to compile the limited-with for it." >>However, 10.1.4(6) which says: >> >> The implementation may require that a compilation unit be legal before >> inserting it into the environment. >> >>is too strict. It could be changed to: > > I don't see why. Again, we need a place for the implementation to "hang" its requirements. We have given them the first handle, by allowing them to require existence. Now we are giving them the second handle, allowing them to impose certain other requirements on the unit. They *clearly* can't impose complete legality requirements, if the units are inserted one at a time, since there is no order of insertion that would allow that. > ... > The GNAT model does not take advantage of this > permission. The permission is there for the benefit of compilers like > Rational, where "inserting" involves running the compiler, and therefore > checking various rules. I suppose AdaMagic takes partial advantage of > this permission, since insertion involves some small amount of syntax > checking. > > But why should we care how much checking is done on B before "limited > with B" can be swallowed by the compiler? If B is illegal, the compiler > can process "limited with B" if it likes, or (if it happens to notice > the illegality), it can complain that "limited with B" is nonsense. > It doesn't matter whether the illegality is syntactic or semantic. You are missing the point. I am simply *allowing* compilers to impose requirements, but only up to a point. Without such verbiage, I don't believe compilers would be allowed to impose any requirements. I suppose you could argue all of this verbiage is unnecessary already, but since we felt the need to put it there in the first place, and we presumably aren't going to take it out, I believe it needs to be updated to *allow* compilers to impose certain requirements on limited-with'ed units. But I definitely don't want to force them to impose these requirements (and I don't think my suggested wording does that). >> The implementation may require that a compilation unit satisfy the Syntax >> Rules before inserting it into the environment. The implementation may >> require that a compilation unit be legal before allowing it to be mentioned >> in a with_clause other than a limited_with_clause. > > I see no need for this added complexity. And it could be a burden -- if > the compiler chooses to implement some simple Legality Rule in the > parser, that should be allowed. I'm not sure what we are discussing any more, since you don't seem to think that the unit even need to exist in the environment to be mentioned in the limited-with clause. I think we need to *allow* implementations to impose some requirements, and we might as well tie it to the notion of "exist" in the environment. *If* we tie it to that notion, we want to be sure that limited-with can in fact be used. That seems to mean that you must be able to insert units into the environment before they can be (fully) compiled, strictly for the purposes of doing a "limited with" of them. Why don't you suggest some wording so we actually have an alternative to consider? ************************************************************** From: Robert A Duff Sent: Monday, February 3, 2003 6:46 PM Tucker wrote: > I think we need to say *something* about the requirements > on a unit mentioned in a limited-with clause. Why not say > it must exist in the environment? As you said, that is > pretty vague. If there are no requirements on it, then > the implementation has no place to "hang" whatever requirements > it really does have. It can't just invent requirements that > don't exist. But it can say, "as far as our implementation > is concerned, that unit doesn't exist in the environment, > so we refuse to compile the limited-with for it." OK, you have convinced me on that point. But as to 10.1.4(6): > Again, we need a place for the implementation to "hang" > its requirements. We have given them the first handle, > by allowing them to require existence. Now we are > giving them the second handle, allowing them to impose > certain other requirements on the unit. But 10.1.4(6) as is already gives sufficient permission. Your proposed change gives *less* permission. I claim there is no need to be more restrictive (on implementations), and that being more restrictive requires more complex RM wording. 10.1.4(6) as is allows compilers to refuse to deal with "limited with X" on an illegal X. That is sufficient permission. If X is in the env, and is legal, the compiler must deal with "limited with X", because the only permission to refuse to admit X into the env is if it's illegal. > They *clearly* > can't impose complete legality requirements, if the > units are inserted one at a time, since there is no > order of insertion that would allow that. There is no requirement that units are inserted one at a time (and in fact they are not, in AdaMagic). Hence, it is not "*clearly*" true that they can't impose legality requirements. Your wording for 10.1.4(6) does not allow a compiler to impose such legality requirements. I think that's overspecification, and could be an implementation burden. For example, it is reasonable to implement some legality rules in the parser, and some compilers do that. One might wish to run the parser on X when seeing "limited with X". Your wording would disallow such an implementation. Imagine a front end structured as three phases: 1. Parse and build syntax tree. 2. Walk tree, build symbol table nodes for explicit declarations, collect a list of all type decls, and check *some* legality rules. 3. Overload resolution, and check the rest of the legality rules. The "limited with" wants that list of type decls from phase 2. But your wording seems to forbid phase 2 from printing any error messages, because they're not syntax errors. > I'm not sure what we are discussing any more, since you > don't seem to think that the unit even need to exist > in the environment to be mentioned in the limited-with > clause. I backed off on that part. >... I think we need to *allow* implementations > to impose some requirements, and we might as well tie > it to the notion of "exist" in the environment. > *If* we tie it to that notion, we want to be sure that > limited-with can in fact be used. That seems to mean > that you must be able to insert units into the environment > before they can be (fully) compiled, strictly for the purposes > of doing a "limited with" of them. > > Why don't you suggest some wording so we actually > have an alternative to consider? OK, I propose changing 10.1.4(5) in the way you suggested, and leaving 10.1.4(6) as is. ************************************************************** From: Tucker Taft Sent: Monday, February 3, 2003 7:02 PM Robert A Duff wrote: > ... > OK, I propose changing 10.1.4(5) in the way you suggested, and leaving > 10.1.4(6) as is. I suppose there is an unwritten rule that there must be some way for two packages with cylic dependence to be inserted into the environment. Clearly if the "insertion" process requires legality, and you can only insert one unit at a time, there is a problem. Either the compiler must allow simultaneous multiple-file insertion (that sounds like it might be illegal in Nebraska ;-), or it must allow units that are not yet demonstrably legal to be inserted. ************************************************************** From: Robert A Duff Sent: Monday, February 3, 2003 7:14 PM Don't you think it's OK to leave that rule unwritten? I mean if that's a hole, it's already a hole. If I have some legal code, I must be able to insert it into the env *somehow*, since there's no permission for the implementation to refuse legal code. The fact that the implementation is incapable of determining whether it's legal seems irrelevant. *I* know it's legal. I think you're worrying overmuch about implementation issues (in considering RM wording). Legal code must be accepted. Therefore, code must be accepted if the compiler does not (yet) know whether it's legal. The compiler can only refuse if it can *prove* it's illegal. Make sense? ************************************************************** From: Tucker Taft Sent: Monday, February 3, 2003 8:33 PM After thinking more about this, I think we need to make some change to the wording of 10.1.4(6). It really doesn't make any sense if we change 10.1.4(5) as proposed. We seem to agree that this sentence is primarily for non-source-based compilers, where the only way to insert a unit into the environment is by processing it in some way. But it is exactly those compilers that will have to change to support limited-with. They *cannot* require legality of all units inserted into the environment, if we have just changed 10.1.4(5) to require that the unit mentioned in a limited-with must already *be* in the environment. I think we should say something closer to this: Before inserting a compilation unit into the environment, the implementation may require that it obey all the Syntax Rules, and any other rules that it can check given what other units already exist in the environment. > ... > I think you're worrying overmuch about implementation issues (in > considering RM wording). Legal code must be accepted. Therefore, code > must be accepted if the compiler does not (yet) know whether it's legal. > The compiler can only refuse if it can *prove* it's illegal. Make > sense? No. ;-). I think either we should make 10.1.4(6) completely implementation-defined, or replace it with something that reflects the problem that non-source-based compilers will have with limited withs, since we know that 10.1.4(6) was crafted specifically for their needs. Remember, this is in the section called "implementation permissions" so it is specifically talking to implementors, not users, and it is specifically talking about implementation issues, not abstract semantic issues. ************************************************************** From: Robert A Duff Sent: Monday, February 3, 2003 7:06 PM Pascal, This "weak with" ("limited with") thing is your proposal, but if you like, I might be willing to work out the RM wording tomorrow (Tuesday). This idea seems to be gaining favor, and if it's to be chosen, I don't really want that to take 17 more ARG meetings. Producing wording now might speed things up. On the other hand, if folks think it's too hard to implement, please say so before I waste my time. Should I do it? ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:21 AM It would be very useful if you did that. You are much more competent than me at crafting RM wording, and furthermore I don't think I would have time to do that between now and the meeting. Since there seems to be some interest for the idea, it would be good to have a precise proposal on the table to discuss it. Experience has proven that discussing an AI without an initial write-up is sterile. ************************************************************** From: Randy Brukardt Sent: Monday, February 3, 2003 8:35 PM I'm not Pascal, but I'd suggest going ahead and writing up the wording. I'm having a hard time getting a handle on this proposal, and I'm sure that would help. OTOH, since it is a complete start from scratch, it's going to take 17 meetings to get it right in any case. At most, you can reduce that number by one. :-) > On the other hand, if folks think it's too hard to implement, please say > so before I waste my time. At the moment, I think it's too hard to implement (given that type stubs are expected to take 4 hours of work; this would be more like 4 weeks). But that may be irrelevant. More important is that I don't see any way, within the current design of Janus/Ada, to implement something like this. The problem is with the obsoleteness check. Janus/Ada uses a serial number created from the time stamp of the symbol table file created for a library unit for checking. Also, the Janus/Ada compiler knows nothing about source files other than the one it is compiling at the moment. (Other tools are a bit smarter, but the compiler does not use that information itself.) It only knows about symbol table files. One presumes that you'd have a new kind of symbol table file that held a 'lightly' compiled package spec for limited withs. And you'd use the time stamp of that file to create the serial number for obsoleteness checking. Clearly, when you 'really' compiled the spec, the new kind of package spec would have to be updated (otherwise, it wouldn't necessarily match the source file, because the source file could have been edited). But that would necessarily change the serial number, and then the links would fail. And there would be no way to avoid the problem. Given that there is no reliable way to determine if a file has been changed (file time stamps are too crude, often changing when nothing actually is changed, and aren't kept to enough accuracy on Windows), I don't see any way to implement this. The compiled=changed model works fine for Ada 95, but is a complete failure for limited with. Essentially, you have to adopt a source-based model, and that is a major change. This also has a giantic impact on build tools. They'd have to be able to process this new clause and come up with an appropriate order in the face of cycles. Of course, tools just punt if that happens now. It may be I just haven't thought about it enough. But I have a lot of things that I need to do before I leave for the meeting, and thinking about this isn't high on the list. ************************************************************** From: Tucker Taft Sent: Monday, February 3, 2003 8:53 PM Randy's comments make me notice something a bit unfortunate about the limited-with proposal. Although we don't need a new kind of compilation unit, we do need a new way of compiling a unit in the non-source-based environment. (I'll call this new way of compiling "preregistering" the unit.) Supporting limited-with means that either the programmer preregisters all compilation units on the off chance they might be mentioned in a limited-with, or the compilation-order- determination tool gets smarter and figures out which units need to be "preregistered" and which don't. Having this additional step could have significant ripple effects into "make" scripts, regression testing scripts, ACATS testing scripts, compilation-order- determination tools, etc. It also means a new step to explain to programmers. None of the above is particularly rocket science, but it does add to the implementation and "deployment" cost of this proposal. In general, I think this might emerge as a very elegant solution, possibly as elegant as the package prefix/abstract approach, but significantly more implementation effort than either the "type C.T;" or the possibly modified/restricted type stub proposal. By the way, although it sounds like implementing "type C.T;" in Randy's compiler might be a bit more work than a restricted type stub, it seems like that is small potatoes compared to the heavy lifting involved in the limited-with proposal. ************************************************************** From: Robert Dewar Sent: Monday, February 3, 2003 8:53 PM BY the way, this discussion of what is and is not required of compilers reminds me that I think it would be good to have an annex for the standard that defined a standard interchange format, right down to the bit-level coding and file format. That would for example, partly satisy Dan's concern about incomaptible source representation issues. ************************************************************** From: Gary Dismukes Sent: Tuesday, February 4, 2003 1:28 AM > But it seems like whatever solution to the mutual recursion problem we > choose, we really need to solve the problem of proliferation of silly > type conversions. Otherwise, these proposals will be nearly unusable. I tend to agree. Certainly with the latest "limited with" proposal there are generally going to be multiple access types for the same designated type and that's true for most of the other proposals as well. It seems though that the models using child packages may be less prone to that since the child package could use an access type declared in the parent and reexport using a subtype if needed. > That is, there ought to be an *implicit* conversion between access types > in the "safe" cases. I know we discussed that before. Is it part of > any of these proposals, or is it a separate AI? I seem to recall that > it might be wrapped up with other access-type issues, like allowing "not > null" constraints, and access-to-constant parameters. Perhaps you're thinking of AIs 230 (generalized access types) and 231 (access-to-constant parameters and null-excluding subtypes). AI-230 would allow more things to be declared using anonymous general access types, which would increase the number of contexts where implicit conversions can occur, though I'm not sure how much help that will be for the mutual-dependence cases where there will be multiple distinct access types and pointers of those distinct types floating around. I suppose it will help to some degree. In any case, I think it's probably best not to wrap a requirement for easier conversions in with the AI-217 proposals, as I think that may tend to muddy the discussion (even more more than it already is;). > > And I do still like that "type C.T;" ;-). > > (what a pain I can be...) > > That's a fine solution, IMHO. Your earlier e-mail about > mutually-recursive types being part of the same abstraction > makes me comfortable with the idea of tying the solution to > child packages. (See, it *is* possible to convey sensible information > about purely aesthetic concerns. ;-)) > > But I would choose the "limited with" idea, unless it is considered too > hard to implement. My feeling as well. Tucker's modified child package proposal definitely seems the simplest both in description and implementation effort (well, I guess Randy doesn't agree with that...), but like others I'm somewhat dissatisfied with requiring a child-based model to get mutual recursion. I was finding my mind being drawn back to wishing we could just do things as simply as other languages are able to, but had decided that such an approach would be tilted too much in favor of source-based compilers. But now that Pascal's bravely put forward that very suggestion, I'm happy to line up behind it if it stands up to scrutiny. I think the critical question is how much work that approach will be for non-source-based implementations. ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 4:04 AM > Perhaps you're thinking of AIs 230 (generalized access types) and 231 > (access-to-constant parameters and null-excluding subtypes). AI-230 > would allow more things to be declared using anonymous general access > types, which would increase the number of contexts where implicit > conversions can occur, though I'm not sure how much help that will be > for the mutual-dependence cases where there will be multiple distinct > access types and pointers of those distinct types floating around. > I suppose it will help to some degree. AI 230 looked quite promising, but it has been on Tuck's action item list for about two years, and he didn't do his homework (but then he'll have some time on the plane, so you never know...) > In any case, I think it's > probably best not to wrap a requirement for easier conversions in with > the AI-217 proposals, as I think that may tend to muddy the discussion > (even more more than it already is;). Agreed. ************************************************************** From: Robert A Duff Sent: Tuesday, February 4, 2003 4:11 PM I don't want to muddy the discussion *too* much. But many (if not all) of the cyclic-dependency solutions are going to introduce extra access type decls. I don't want folks to reject proposals for that reason. Therefore, we need to at least keep an open mind -- assume that the access-type-conversion annoyance can be solved. ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:02 AM > In Pascal's proposal, a parent package P can weakly 'with' its children: > "weak with P.C;", if you want a mutually dependent pair of types, > one in P and one in P.C. The spec of P will be elaborated before the > spec of P.C, as usual. > > However, it is pointless for P.C to say "weak with P;", because the > child-to-parent dependency is necessarily strong (given that there's no > proposal to add syntax saying otherwise -- "weak children?"). > > No problems here. I'm just making some observations. In my proposal it would also be possible for unit P to say "limited with P", but I think we would want to forbid that, as it seems methodologically dubious. (If you follow the model to its logical conclusions, it would for instance allow forward references to packages and types within the spec of P.) ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:21 AM > The first is where you have a type declaration that depends on another > type: > > type Color is (Red, Green, Blue); > type Properties(C: Color) is...; > > Does the incomplete type for Properties include the discriminant? I have no doubt that the incomplete type for Properties doesn't include the discriminant. We discussed that issue when we did "with type" and we concluded that any other choice was causing endless trouble. Discriminants of incomplete types are not too useful anyway, they can only be used in building contrained access-to-object types. And the added complexity is not worth it. > I prefer the choice where only private, record and tagged > types are incomplete in the limited view. But then you would need to do a lot of semantic analysis, and you would have to define very precisely what happens during a "superficial" compilation. Consider for instance: type R is record ... end record; type T is new Integer range R'Alignment .. R'Size; How can T be properly defined if R is incomplete? And don't tell me that T would be some new kind of "incomplete integer type", because that would add so much complexity to the language that I'd rather forget about the entire proposal. As Tuck pointed out in his "willies" message, the only way that this model can be made to work is if the first phase can be performed by a brain-dead parser. ************************************************************** From: Robert I. Eachus Sent: Tuesday, February 4, 2003 11:38 AM Pascal Leroy wrote: >I have no doubt that the incomplete type for Properties doesn't include the >discriminant. We discussed that issue when we did "with type" and we concluded >that any other choice was causing endless trouble. About 4 AM, I realized that what is needed is the fact that Properties is a type with discriminants, notthing more. But we already have that concept well defined, and it covers incomplete tagged types. Then I went back to sleep. So I propose that (incomplete) tagged types and types like Properties have unknown discriminants. This allows incomplete discrete types to be treated like other incomplete types, and incomplete array types to also be treated as incomplete with unknown discriminants. >But then you would need to do a lot of semantic analysis, and you would have to >define very precisely what happens during a "superficial" compilation. Consider >for instance: > > type R is record ... end record; > type T is new Integer range R'Alignment .. R'Size; > >How can T be properly defined if R is incomplete? And don't tell me that T >would be some new kind of "incomplete integer type", because that would add so >much complexity to the language that I'd rather forget about the entire >proposal. The nightmare that woke me up was not quite this case, but related. (Using T'First, etc., of another discrete type.Hiding the literals is not enough.) As for access types, I still think we need to special case those. At first I was only worried about the proliferation of access types with the same target. But then I realized that the type matching nightmare gets worse in cases where some of the potential access types are hidden by incompleteness. The messy case is X: T := F.all. F can be an overloaded function, and I would hate to have the legality checked twice, especially if one of the checks depended on elaboration order. There is another painful case though, and I am not sure what to do about it... limited with B; package A is type Foo is record...end record; ... end A; with A; package B is type TA is access A.Foo; type Bar is record TAC: TA; end record; ... end B; (You can produce variations on this theme by creating a third package in the cycle.) We are not talking pathological here, this is part of the core functionality. Now, what is the status of B.TA inside the spec of A? If it is incomplete we get access type proliferation. If it is an access type, is it an incomplete access type? Inquiring minds want to know. If you prefer, write the example as: limited with A.B; package A is type TA is access B.T; ... end A; package A.B is type T is tagged record...end record; ... end A.B; I can live with this version working where the first one breaks, but now we get back to the issue of representation of the access type. Whether it has to be a fat pointer or not will depend on features of T. Again, I can live with that, we just have to determine which features are visible. (Unknown discriminants for T in this case seems minimal.) ************************************************************** From: Tucker Taft Sent: Tuesday, February 4, 2003 2:20 PM "Robert I. Eachus" wrote: > About 4 AM, I realized that what is needed is the fact that Properties > is a type with discriminants, notthing more. Why? >... But we already have that >concept well defined, and it covers incomplete tagged types. Then I went >back to sleep. So I propose that (incomplete) tagged types and types >like Properties have unknown discriminants. This allows incomplete >discrete types to be treated like other incomplete types, and incomplete >array types to also be treated as incomplete with unknown discriminants. Why can't we treat these all as plain old incomplete types? You haven't explained that. >>But then you would need to do a lot of semantic analysis, and you would have to >>define very precisely what happens during a "superficial" compilation. Consider >>for instance: >> >> type R is record ... end record; >> type T is new Integer range R'Alignment .. R'Size; >> >>How can T be properly defined if R is incomplete? And don't tell me that T >>would be some new kind of "incomplete integer type", because that would add so >>much complexity to the language that I'd rather forget about the entire >>proposal. >> >The nightmare that woke me up was not quite this case, but related. > (Using T'First, etc., of another discrete type.Hiding the literals is >not enough.) I am totally lost. These types are incomplete. You can't do anything with them. > As for access types, I still think we need to special case those. At > first I was only worried about the proliferation of access types with > the same target. But then I realized that the type matching nightmare > gets worse in cases where some of the potential access types are hidden > by incompleteness. The messy case is X: T := F.all. F can be an > overloaded function, and I would hate to have the legality checked > twice, especially if one of the checks depended on elaboration order. I must be missing something fundamental here. All of these types are just plain old incomplete types. Yes, I agree we ought to special case tagged and allow a bit more, but that seems irrelevant to what you are talking about. What is messy about the messy case? >... >(You can produce variations on this theme by creating a third package in >the cycle.) We are not talking pathological here, this is part of the >core functionality. Now, what is the status of B.TA inside the spec of >A? If it is incomplete we get access type proliferation. If it is an >access type, is it an incomplete access type? Inquiring minds want to know. It is incomplete and we get access type proliferation. I don't see that "limited with" can solve the access type proliferation problem, and trying to make it do so is doomed, in my mind. ... >I can live with this version working where the first one breaks, but now >we get back to the issue of representation of the access type. Whether >it has to be a fat pointer or not will depend on features of T. Again, >I can live with that, we just have to determine which features are >visible. (Unknown discriminants for T in this case seems minimal.) I am presuming that all non-tagged types are incomplete, period. No new notion of "incomplete access types" is being added. ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:00 PM > > So I propose that (incomplete) tagged types and types > > like Properties have unknown discriminants. This allows incomplete > > discrete types to be treated like other incomplete types, and incomplete > > array types to also be treated as incomplete with unknown discriminants. > > Why can't we treat these all as plain old incomplete types? > You haven't explained that. I am as puzzled as Tuck. It is my understanding that unknown discriminants for incomplete types are pretty much equivalent to no discriminants. So it's fine with me to insist that these types have unknown discriminants, but it doesn't make the slightest difference. (Remember, we are talking _incomplete_ types here, so you cannot do much with them anyway.) > I am presuming that all non-tagged types are incomplete, period. No new > notion of "incomplete access types" is being added. Agreed. This seems like the only way to avoid madness. Granted, this proposal doesn't solve the problem of proliferation of access types, but that problem shows up in contexts that are unrelated to circular type dependencies, so I think that it should be addressed by a different mechanism. ************************************************************** From: Randy Brukardt Sent: Tuesday, February 4, 2003 2:53 PM Tucker said, replying to Robert Eachus: > > If it is incomplete we get access type proliferation. If it is an > > access type, is it an incomplete access type? Inquiring > > minds want to know. > > It is incomplete and we get access type proliferation. > > I don't see that "limited with" can solve the access type proliferation > problem, and trying to make it do so is doomed, in my mind. At which point, I lose interest in the proposal. All of type stubs, restricted type stubs, and Tuck's type C.T idea do solve the access type proliferation problem. Admittedly the structure of the declarations is unnatural. But the only real alternative is to adopt something like AI-230. That bulks up the proposal a ton. So, limited with: -- Is much harder to implement in library based compilers. It's probably harder to implement than type stubs/type C.T in source-based compilers as well (certainly no easier). Affects compilation tools in all environments; -- Doesn't solve the access type proliferation problem. The only advantage seems to be: -- A more natural expression of package structure. But that's dubious, since you need to resort to unnatural package structures in order to share access types. So you're pretty much left with "much harder to implement". ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:09 PM > > I don't see that "limited with" can solve the access type proliferation > > problem, and trying to make it do so is doomed, in my mind. > > At which point, I lose interest in the proposal. Access type proliferation in and of itself is not a problem. What is a problem is access type conversions. That problem shows up in all sorts of contexts, notably in OOP where you have access types (class-wide or specific) designating some type in a hierarchy, and all conversions must be explicit, even those that cannot possibly fail. That is a big pain. This has nothing to do with circular type dependences. Even if we select a solution to the problem of circular type dependences that doesn't cause proliferation of access types, the problem of access type conversions will remain. In C, if you have a type t you automatically get a pointer type t*. In Ada, I often wish we had a similar capability, where I wouldn't have to declare all these silly access types and convert between them explicitly. ************************************************************** From: Robert I. Eachus Sent: Tuesday, February 4, 2003 5:20 PM Tucker Taft wrote: >It is incomplete and we get access type proliferation. > >I don't see that "limited with" can solve the access type proliferation >problem, and trying to make it do so is doomed, in my mind. And in my mind the proposal is doomed (or not the best one on the table) if it doesn't address the access type proliferation problem. >I am presuming that all non-tagged types are incomplete, period. No new >notion of "incomplete access types" is being added. Hmm. I must not be making myself clear. My idea is not to add "incomplete access types," but to try to make the limited with idea work with access types to these new special incomplete types. The refractory issue is when you have access types that carry descriptor information and when you don't. Where that data is stored is irrelevant. Even if you store the descriptor for an unconstrained array type with the data and pointers to both unconstrained and constrained arrays are the same size, some conversions and assignments are impossible or require recopying the data: package P is type Str is new String; subtype Str4 is Str(1..4); type PT1 is access Str; type PT2 is access Str4; end P; or if you prefer: limited with P; package Q is type PT3 is access P.Str; type PT4 is access P.Str4; end Q; Now I need to compile some code--perhaps the body of Q--that needs to do those damned explicit type conversions, or even just some simple assignments. As a programmer I have a problem, but without more information about the types in P than Tuck seems willing to allow, the compiler is SOL. It doesn't know how to create objects of type PT3 and/or PT4, and even has trouble with legality issues: ... PO3: PT3; PO4: PT4; ... PO3.all := PO4.all; -- legal? Right now the voting seems to be that either the limited with idea is doomed because it is too much work, or it is doomed because it can't deal with the issue of access type proliferation. I'll let sleeping dogs lie. ************************************************************** From: Tucker Taft Sent: Tuesday, February 4, 2003 5:58 PM "Robert I. Eachus" wrote: > > Tucker Taft wrote: > > >It is incomplete and we get access type proliferation. > > > >I don't see that "limited with" can solve the access type proliferation > >problem, and trying to make it do so is doomed, in my mind. > > > And in my mind the proposal is doomed (or not the best one on the table) > if it doesn't address the access type proliferation problem. Well, I think trying to make it address this problem very tricky, since it means looking at the designated subtype indication, and that will get you back to worrying about use visibility, etc. > >I am presuming that all non-tagged types are incomplete, period. No new > >notion of "incomplete access types" is being added. > > > Hmm. I must not be making myself clear. My idea is not to add > "incomplete access types," but to try to make the limited with idea work > with access types to these new special incomplete types. The refractory > issue is when you have access types that carry descriptor information > and when you don't. Where that data is stored is irrelevant. Even if > you store the descriptor for an unconstrained array type with the data > and pointers to both unconstrained and constrained arrays are the same > size, some conversions and assignments are impossible or require > recopying the data: I am getting an inkling of your concern. At least one of the problems you seem to be worrying about is an implementation issue (as opposed to a semantics issue), and is one that bedevils the "unrestricted" type stub proposal, and the "with type" proposal. Namely that when choosing a representation for access types to the full type, the compiler may not be aware that there are access types to the incomplete type. I don't think this is quite as serious as the problem with the "with type" proposal, since the compiler is doing some serious peeking at the full type when it creates the incomplete type, so *if* it has multiple access type representations, it can mark the incomplete type with enough extra information so that it will choose an appropriate representation for any access type that is declared later which "sees" only the incomplete type. Unfortunately, derived types create a bit of a problem, since they don't reveal syntactically what sort of type they are (e.g., whether their first subtype is an unconstrained array subtype and hence pointers to it should be default be "fat" pointers). This would mean that the "peeker" would have to make a "guess" about whether the incomplete type is an unconstrained array. Probably it would assume all derived types are not unconstrained arrays. In any case, whatever guess it makes, it would want to record that in the limited view of the package. When the full view is compiled, as part of the "compatibility" check it would also have to check that the "guess" was correct. If the guess was wrong, and the type is in fact an unconstrained array, say, then it would have to record that fact on the incomplete type, and bump the timestamp (or equivalent) of the limited view of the package. Ultimately the units that depend on this limited view would get recompiled. For a source-based compiler, there isn't really any place to permanently record the "correct" answer, so it would probably end up being a link-time check, and if there were an incompatibility, the offending units would be recompiled with a switch directing them to treat the problematic incomplete type as an unconstrained array, even though it wasn't obvious. > > package P is > type Str is new String; > subtype Str4 is Str(1..4); > type PT1 is access Str; > type PT2 is access Str4; > end P; > > or if you prefer: > > limited with P; > package Q is > type PT3 is access P.Str; > type PT4 is access P.Str4; This wouldn't be legal, since Str4 is defined by a subtype, not a type declaration. > end Q; > > Now I need to compile some code--perhaps the body of Q--that needs to do > those damned explicit type conversions, or even just some simple > assignments. As a programmer I have a problem, but without more > information about the types in P than Tuck seems willing to allow, the > compiler is SOL. It doesn't know how to create objects of type PT3 > and/or PT4, and even has trouble with legality issues: I don't see how legality issues enter into the problem of access type representation. You can't do much of anything with incomplete types. And you can't dereference access-to-incomplete, unless perhaps there is a complete type "nearby". > ... > PO3: PT3; > PO4: PT4; > ... > PO3.all := PO4.all; -- legal? > > Right now the voting seems to be that either the limited with idea is > doomed because it is too much work, or it is doomed because it can't > deal with the issue of access type proliferation. I'll let sleeping > dogs lie. I don't think it has been abandoned, so it is still important to try to identify (real ;-) problems associated with the proposal. ************************************************************** From: Robert I. Eachus Sent: Tuesday, February 4, 2003 6:34 PM Tucker Taft wrote: >Well, I think trying to make it address this problem very tricky, >since it means looking at the designated subtype indication, >and that will get you back to worrying about use visibility, etc. Yep. ... >For a source-based compiler, there isn't really any place to >permanently record the "correct" answer, so it would probably >end up being a link-time check, and if there were an incompatibility, >the offending units would be recompiled with a switch directing >them to treat the problematic incomplete type as an unconstrained array, >even though it wasn't obvious. Yep, it is a problem in any solution to the mutual dependence problem. The issue here is deciding what aggrevations to add--to the user, the implementor, or both. We could make fairly draconian rules that stated that any unit that actually uses a subtype, access type to, or type derived from one of these magic incomplete types must depend on the limited withed unit, and thus see the full type. I think so far this has been an implicit assumption, but the devil is in the details. To take your discussion above a bit further, what happens when you modify the limited withed unit in a way that changes the necessary representation of an access type declared elsewhere? It begins to look like a very real dependence somehow magically wished away. You end up having to recompile some units due to changes in units they don't depend on.... >I don't think it has been abandoned, so it is still important >to try to identify (real ;-) problems associated with the >proposal. > Consider the (dead) horse to have been additionally flogged. ************************************************************** From: Robert A Duff Sent: Tuesday, February 4, 2003 7:06 PM Note that we already have a case where you can create access-to-incomplete without knowing what the complete type looks like. Namely, an incomplete type declared in a package spec, and completed in the body. ************************************************************** From: Robert I. Eachus Sent: Tuesday, February 4, 2003 10:34 PM What did we used to call that? Ah, yes, I remember the Tucker Taft amendment. ;-) The only good thing about it was that the incomplete type had to be private, so it couldn't be seen outside the package. If you weren't following LMC/ARG actions then look up (Ada-83) AI-7. For one meeting, John Goodenough put the AI's for consideration in two books--AI-2 and AI-7, plus related AI's, and everything else. (The final version of AI-00007/19-BI-WJ--yes that is version 19--is actually a consolidation of about a dozen separate AI's) As I recall, AI-7 underwent combinatorial explosion, we discovered lots of nits and crannies, and finally came up with a smoking gun case that did not require the TTA to fire off. But if you think my concern about access to incomplete types with discriminants is overdone, you really should go back and look at all those AI's. As I recall, the final version of AI-7 allowed the legality check on discriminant constraints of access to incomplete objects to be done in one of three places--because there were cases when each of the three wouldn't work, and in some cases more than one. Any "solution" to the interlocking types problem that re-opens that can of worms should be shot then burned and the ashes stirred and buried under a crossroads at midnight. I think net that AI-7 and its relatives took the equivalent of several three-day meetings to resolve. (This was "fixed" in Ada 95 by deferring all compatibility checks until object creation. ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:55 AM > At the moment, I think it's too hard to implement (given that type stubs are > expected to take 4 hours of work; this would be more like 4 weeks). But that > may be irrelevant. More important is that I don't see any way, within the > current design of Janus/Ada, to implement something like this. Not sure how you came up with the 4 hours estimate, but I'm sure it would take me more than that to do a detailed design. At this point I'd say that the implementation effort for any of these proposals would be in the range 4 weeks .. 4 months, and I can't be more specific without doing a detailed analysis. But anyway, I don't think that numbers are very interesting at this point. It's really the essence of the implementation difficulties that need to be looked at. As far as I can tell, you and I have the same set of problems, because we are both library-based. I realize that the devil is in the details, and that the magnitude of changes can vary substantially from one implementation to another, but I don't buy that the "limited with" proposal would force you to go to a source-based model. (For Rational, going to a source-based model is a no-no; we'd rather get out of the Ada business; so hopefully there has to be an implementation technique where we stick to our current library model.) > The problem is with the obsoleteness check. Janus/Ada uses a serial number > created from the time stamp of the symbol table file created for a library > unit for checking. Also, the Janus/Ada compiler knows nothing about source > files other than the one it is compiling at the moment. (Other tools are a > bit smarter, but the compiler does not use that information itself.) It only > knows about symbol table files. Fine. If I substitute "Diana tree" for "symbol table", that's essentially true of Apex too. (Ignoring all the incremental compilation crap.) > One presumes that you'd have a new kind of symbol table file that held a > 'lightly' compiled package spec for limited withs. And you'd use the time > stamp of that file to create the serial number for obsoleteness checking. > Clearly, when you 'really' compiled the spec, the new kind of package spec > would have to be updated (otherwise, it wouldn't necessarily match the > source file, because the source file could have been edited). And we would indeed have the same problem. We would first created a "parsed" Diana tree (during "superficial" compilation) and then we would rewrite it to make it "analyzed" (during "full" compilation). Surely that would change the timestamp. > But that would > necessarily change the serial number, and then the links would fail. And > there would be no way to avoid the problem. > > Given that there is no reliable way to determine if a file has been changed The way that I plan to do this is to store in the "parsed" Diana tree (what you call the "light symbol table") a hash code. This hash code could be based on the sequence of tokens in the source, or it could be more clever and be based only on the names of types and packages (the information that is available after "superficial" compilation). Whenever clients currently use a timestamp to check for obsolescence, they would use the combination . If the timestamps match, everything is fine. Otherwise, the Diana trees need to be opened, and the hashes need to be compared. I realize that this will result in some extra open system calls, but Ada's separate compilation already requires a zillion system calls, so I won't lose any sleep on that. Of course, a hash code is not exactly 100% safe, as there could be collisions. But if you design it well enough, you won't see a collision in your lifetime. (We use 96-bit hash codes all over the place for incremental compilation, and we have never run into a collision in all these years.) > This also has a giantic impact on build tools. They'd have to be able to > process this new clause and come up with an appropriate order in the face of > cycles. Of course, tools just punt if that happens now. But the trick is that there are no prerequisites for "superficial" compilation. So one approach is to blindly run "superficial" compilation over all the units, in any order you like (e.g. the order of i-nodes in the filesystem if you like). And then run "full" compilation in an order compatible with normal with clauses and other semantic dependencies, just like you do now (and ignoring "limited with" clauses). No need to process the new clause, no need to change the algorithm that determines the compilation order. Of course you might want to be more clever and avoid the "superficial" phase if it's not needed, but that's an optimization, not something you would have to do for correctness. ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 4:00 AM > Supporting limited-with > means that either the programmer preregisters > all compilation units on the off chance they might > be mentioned in a limited-with, or the compilation-order- > determination tool gets smarter and figures out which > units need to be "preregistered" and which don't. As I mentioned in my previous response to Randy, it would be just fine if the compilation-order-determination tool were to systematically preregister each and every unit, and then run the normal ordering algorithm, ignoring "limited with" clauses. I would think that compiler writers would prefer to go that way, rather than breaking each and every compilation script in the universe. So I am arguing that the new step should be essentially invisible to programmers. In an implementation where the preregistration step would be costly, then yes, the ordering algorithm might have to be modified to avoid preregistration when possible. ************************************************************** From: Erhard Ploedereder Sent: Tuesday, February 4, 2003 7:38 AM I like "limited with". It is the closest we got to what the user expects. I would like to see a writeup. (I have had bellyaches with Tuck's proposal of tying the cyclic dependency to child packages. I waited for the aches to go away. They did not.) ************************************************************** From: Randy Brukardt Sent: Tuesday, February 4, 2003 3:00 PM > > At the moment, I think it's too hard to implement (given that type stubs are > > expected to take 4 hours of work; this would be more like 4 weeks). But that > > may be irrelevant. More important is that I don't see any way, within the > > current design of Janus/Ada, to implement something like this. > > Not sure how you came up with the 4 hours estimate, but I'm sure it would take > me more than that to do a detailed design. I did a detailed design (in my head) when I was unable to sleep Saturday night. > At this point I'd say that the implementation effort for any of these proposals > would be in the range 4 weeks .. 4 months, and I can't be more specific without > doing a detailed analysis. But anyway, I don't think that numbers are very > interesting at this point. It's really the essence of the implementation > difficulties that need to be looked at. The reason I was thinking about it was that the restricted type stubs model pretty much matches how Janus/Ada works internally anyway. So the main cost is "connecting" the stub to the completion. (And many stubs works, because it's a one-way pointer. But I wouldn't expect that to be true on other implementations.) Since incomplete and private types use the same code in most circumstances, the visibility implications of "availability" should be free. (That is the biggest assumption, of course, because there are no such rules in Ada 95.) So all of the work is doing the connection, and a brute force version (walk all of the symbol table looking for stubs, then when a stub is found, walk the whole symbol table looking for a connection) would only need to be called in one place. Most of the work is figuring out where to call that routine. A better version would save the stubs in a list as they are loaded, so we wouldn't have to search for them. That version would have no distributed cost and not a lot of cost even when stubs are used, so I doubt its possible to do better. Tucker's C.T adds a lot of messing around with ghost packages, but otherwise is identical (we don't have to look quite so far for a completion, but the process is the same). Which is why I'm certain its more work. The current stub stubs requires two stubs of the same completion to match, even when the completion isn't available. That's more work, but I don't think it's a very significant amount (it's just another weird special case in the type matcher, just like anonymous access types and T'Class). In all of these cases, I wouldn't be surprised to find glitches, but that would require a pretty complete test suite. Building that would take longer than doing the implementation (that is often true). Limited with would require a new kind of symbol file, or something (Pascal suggests a hash). It also would require going in an adding a bunch of stuff to the make tool. No one understands how that thing really works, so that alone is a daunting project. And the failure to provide any sort of solution to the access type problem makes it feel like "with type" all over again (which isn't surprising, because it *IS* essentially "with type", just as "type C.T" is just a different syntax for type stubs). ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:17 PM > Since incomplete and private types use the same code in > most circumstances, the visibility implications of "availability" should be > free. Aren't you concerned about types having three views (incomplete, partial, full) in the case of an incomplete type completed by a private type? That gives me the willies, to borrow Tuck's words, because it's currently very complicated in our compiler to decide what properties of a type are visible at a given place (because of the separation between partial and full view, because of characteristics that become visible "later within the immediate scope", etc.). Adding a third view is not going to make this easier. ************************************************************** From: Randy Brukardt Sent: Tuesday, February 4, 2003 3:43 PM No, because we have separate type records for each view, and they have their own visibility. For type matching purposes, the only question is whether you can or cannot walk the pointer to the 'next' view, as we always match (and use for other purposes) the "fullest" view available. That depends on visibility, etc. (We have a separate set of routines that always goes to the completion, which the intermediate code generation uses. To heck with visibility at that point.) Since this stuff is recursive (well, actually it uses a loop), you'd first try to go from the incomplete block to the private block (depending on the visibility of the private), and then, once you had the private, you'd do the checks that we currently have. So I don't see a problem with three parts or ten parts for that matter. But I realize that if you somehow managed to implement types with a single record for both views (we tried that and concluded it was impossible. But perhaps we weren't clever enough. Or perhaps its impossible because of shared generics.), it might be much worse. But I think that any proposal has to allow stubs (or whatever) of private types. So you're going to have the three view issues, and if that is a problem, any solution to this problem will be a nightmare to implement. In which case you ought to stay out of the implementation difficulty debate altogether. :-) :-) ************************************************************** From: Gary Dismukes Sent: Tuesday, February 4, 2003 3:20 PM Pascal wrote: > In my proposal it would also be possible for unit P to say "limited with P", but > I think we would want to forbid that, as it seems methodologically dubious. (If > you follow the model to its logical conclusions, it would for instance allow > forward references to packages and types within the spec of P.) One question is, what does it mean when you say "limited with P.C.D". Is it equivalent to having limited with for P, P.C, and P.C.D, in analogy with normal with clauses? That was my first thought, and it seems natural to make it behave the same, but perhaps it's more reasonable to say it only applies to the final named unit. That would avoid the issue of self-circular references when a parent withs its child and we would also want the rules to disallow a direct limited with of yourself. Another small issue is whether use clauses are allowed for packages named in these with clauses. I imagine that the implementation is effectively going to create a package entity containing an incomplete type representative for each type in the real package's visible part (along with representatives for any nested packages presumably) and since this will look essentially the same as other packages in the symbol table environment I don't see any technical problem with permitting a use clause for these packages. Perhaps there are methodological reasons for not allowing it, though I can't think of any. Changing a limited with to a normal with could introduce new illegalities due to name clashes, but that's no worse than adding a new with and use clause for some package. ************************************************************** From: Pascal Leroy Sent: Tuesday, February 4, 2003 3:30 PM > One question is, what does it mean when you say "limited with P.C.D". > Is it equivalent to having limited with for P, P.C, and P.C.D, in > analogy with normal with clauses? That was my first thought, and > it seems natural to make it behave the same, but perhaps it's more > reasonable to say it only applies to the final named unit. That > would avoid the issue of self-circular references when a parent > withs its child and we would also want the rules to disallow > a direct limited with of yourself. From a pedagogical standpoint it would seem simpler to say that "limited with" works like "with", i.e. that "limited with P.C.D" is equivalent to "with P, P.C, P.C.D". Any other option is going to unnecessary confuse users. So maybe the right answer is that if a package has a "limited with" of itself it has no effect (but is not illegal). > Another small issue is whether use clauses are allowed for packages > named in these with clauses. I imagine that the implementation is > effectively going to create a package entity containing an incomplete > type representative for each type in the real package's visible part > (along with representatives for any nested packages presumably) > and since this will look essentially the same as other packages > in the symbol table environment I don't see any technical problem > with permitting a use clause for these packages. Yes, I was thinking that use clauses and package renamings would be OK in that context, although if they lead to problems we could disallow them. But I can't think of any problem at this point. ************************************************************** From: Robert Dewar Sent: Tuesday, February 4, 2003 10:19 PM > In C, if you have a type t you automatically get a pointer type t*. In Ada, > I often wish we had a similar capability, where I wouldn't have to declare > all these silly access types and convert between them explicitly. For some reason, I never really understood it, this proposal, which I was strongly in favor of, never got significant support (I had suggested early on calling it type'access). ************************************************************** From: Tucker Taft Sent: Tuesday, February 4, 2003 3:27 PM In discussing this further with Bob Duff, and mulling it over more in my mind, I see the following restrictions would probably be needed on the "limited with" capability. I am using the term "limited view" of a package to be what you get when you mention a package in a limited-with-clause. I am presuming that a limited view of a package contains only nested packages and types, and all the types are [tagged] incomple. A "limited with" *cannot* make the following declarations visible: 1) Package instantiation Because the generic being instantiated needs to be visible if we are to determine what types it contains, and identifying the generic may require resolving a name that is only use-visible, or that is from some other compilation unit 2) Package renaming Because the package being renamed needs to be visible if we are to determine what types it contains, and identifying the renamed package may require resolving a name that is only use-visible, or that is from some other compilation unit A "limited view" of a package *cannot* be mentioned in: 3) A "use" clause Because we can get a funky kind of Beaujolais effect if we have a "use" clause for two different packages, and due to changing a "normal" with clauses inherited from some ancestor unit the meaning of an identifier switches from one thing to another; e.g.: package A is X : Integer := 7; end A; package B is X : Integer := 203; end B; with A; -- change this to "with B;" and see what happens package P is end; limited with A, B; use A; use B; -- Probably shouldn't be allowed if only have limited view package P.C is Y : Integer := X; -- which X is this? end P.C; This is presuming that if we were to allow a "use" clause for a limited view of a package, it would make only the types and subpackages directly visible. If the "use" clause made everything in the package visible part visible, but only the types were actually "usable" that could work, but having visible but unusable declarations of *all* kinds, rather than just types and packages, could significantly increase the effort of building up a limited view. 4) A package renaming declaration This restriction is probably not as critical as the others. The problem comes when someone from outside the unit containing the limited with references this package renaming. What does it see? If it has a "regular" with for the target of the limited-with, does it see the "full" view of the package via the renaming, or only the limited view of it. Possible restriction (4) brings up the issue of the opposite situation, where a given unit has a limited view on a package, but also has visibility on a renaming of a full view of the package. There seem to be various possibilities: a) The renaming also provides only a limited view when referenced from the given unit. b) The given unit has a full view of the package, through either the renaming or the name introduced by the limited-with clause c) The given unit has a limited view via the name introduced by the limited-with clause, and a full view via the renaming, and they are essentially completely unrelated packages. d) (c), except the packages are recognized as different views of the same package, and the incomplete types in the limited view are recognized as being completed in the renaming, and so are treated as non-limited types for all intents and purposes. (a) is probably the most consistent with the way package renamings work now, in that what children you see via a renaming of a library unit package is determined by the "with" clauses in the unit referencing the renaming, rather than by the with-clauses in the unit containing the renaming. (b) and (d) are both similar to the choice I suggested for "type C.T;" where if the completing type declaration is visible (including presumably via a renaming), then the incomplete type declaration is hidden from all visibility. In the "type C.T;" proposal, this presumably implies that it is as though the "type C.T;" declaration were not there at all, so if a renaming "package R renames C;" is visible, but "C" itself is not visible (because C was not directly "withed"), then you *cannot* refer to the type at all via "C.T." You can refer to it via "R.T" and then of course it is a full type. There are other possible ways of handling this for "type C.T;" but this approach required the smallest change to 18.3(19) and at least makes some kind of sense. For all proposals, the issue of visible renamings of the enclosing package when the package itself is not visible is thorny. A related question is when the completing type is *not* visible (including not via a renaming), but a declaration of a subtype *is* visible, or an object of the type is visible. The questions are always: i) Is the type named via the "limited view" name complete or incomplete? ii) If incomplete, does it nonetheless "match" the complete type in certain contexts? To put it more concretely: package P1 is type A is access ; -- incomplete view is visible via a limited with, type stub, type C.T, or whatever Y, Z : A; end; package P2 is type T is ; end P2; with P2; package P3 is X : ; end P3; with P1, P3; -- No "with" of P2, no visibility on P2.T procedure P4 is begin ... P3.X := P1.Y.all; -- when is this legal? (*) P1.Y.all := P3.X; -- same question (**) if P1.Y.all = P1.Z.all then -- same question (***) ... Hopefully the answers to (*) and (**) are the same, though the asymmetric Name Resolution wording of 5.2(4) on assignment statements seems like (*) and (**) might be treated differently unless we are careful. Note that last time we worked on the "availability" rules for type stubs, we required that either we be in the scope of a with clause for the enclosing package (or a renaming thereof, I presume), or we be in a context where we have the complete type "nearby" (I forget the wording at this point). This would say that having a visible (but not "with"ed) renaming of the enclosing package, or having a visible subtype, wouldn't help. For the above cases, this would mean that probably (*) and (**) would be legal, but (***) would not be legal. Have a nice day... ;-) ************************************************************** From: Tucker Taft Sent: Tuesday, February 4, 2003 3:48 PM Pascal Leroy wrote: > Randy wrote: > > One presumes that you'd have a new kind of symbol table file that held a > > 'lightly' compiled package spec for limited withs. And you'd use the time > > stamp of that file to create the serial number for obsoleteness checking. > > Clearly, when you 'really' compiled the spec, the new kind of package spec > > would have to be updated (otherwise, it wouldn't necessarily match the > > source file, because the source file could have been edited). > > And we would indeed have the same problem. We would first created a "parsed" > Diana tree (during "superficial" compilation) and then we would rewrite it to > make it "analyzed" (during "full" compilation). Surely that would change the > timestamp... After talking with Bob Duff about this a bit, I don't think this really works. I think you need to keep around indefinitely both a "full view" of the package and a "limited view" of the package. Even after you do a "full" compile, some units can still do "limited with" and should only see the limited view. What I would recommend is you treat them as pretty much distinct units. When you fully compile, if there is a limited view already present, you could check that it is compatible with the full view (i.e. has the same names in the package/type tree, and the same types are tagged). I think if they are not compatible, then and only then does it seem worth obsoleting the limited view and any units compiled against it. There seems no point in hashing the incomplete view, since it is easy enough to check compatibility. Hashing would only be an optimization if this compatibility check turned out to be really expensive. If a unit requests a limited view, and the only non-obsolete thing you have around is the full view, I would at that point create the limited view from the full view. I would bump the time stamp on the limited view only when it is created or updated, and units compiled against the limited view only worry about that time stamp. The time stamp of the full view is irrelevant to them. ************************************************************** From: Randy Brukardt Sent: Tuesday, February 4, 2003 4:13 PM "Easy enough to check compatibility"? You've got to be kidding. The only thing that we know how to do with a symbol file is load them into a symbol table. But you can't do that to "check compatibility", because you've got to load the full view into the same place (can't have two packages with the same library-level name). Pascal's hash idea (instead of a time stamp) makes much more sense. You'd create the limited symbol file for every compilation (full and limited) that you did, but the hash would only change if it actually is different. Keep in mind that no one actually "obsoletes" anything. How we do obsoleteness checking is simply to compare the time stamp serial numbers of every unit withed transitively for every unit withed. They better all match. Any mismatch is reported as an error, and the compilation aborted. And we of course repeat the check when linking. We use the order only to provide better error messages, and units are NEVER removed from the program library (unless the programmer does so manually). ************************************************************** From: Tucker Taft Sent: Tuesday, February 4, 2003 5:05 PM > "Easy enough to check compatibility"? You've got to be kidding. The only > thing that we know how to do with a symbol file is load them into a symbol > table. But you can't do that to "check compatibility", because you've got to > load the full view into the same place (can't have two packages with the > same library-level name). This is my whole point. You *do* need to have both views available at once, since one unit might have a "limited with" of package P, while some unit that it depends on indirectly has a full "with" of P. To avoid ripple effects, we want the one with the limited-with to only see the limited view. You mentioned in another note that you have separate symbol table entries for various different forms of a type. I think you will need separate symbol table entries for the limited view and the full view. In other words, for a library package P, you really have two units, one whose library-level name is "P-full" and one whose name is "P-limited". > Keep in mind that no one actually "obsoletes" anything. How we do > obsoleteness checking is simply to compare the time stamp serial numbers of > every unit withed transitively for every unit withed. They better all match. Yes, I understand that approach. I probably should have simply talked in terms of bumping time stamps. I think the same point can be made in those terms. We have a similar model, albeit only in the program library we build up in memory. [Since we allow the compiler to run for a "long" time and to process different versions of the same file during a single execution, we have all the obsoleteness checking mechanism in there as well.] ************************************************************** From: Randy Brukardt Sent: Tuesday, February 4, 2003 5:34 PM > This is my whole point. You *do* need to have both views available > at once, since one unit might have a "limited with" of package P, > while some unit that it depends on indirectly has a full "with" of P. > To avoid ripple effects, we want the one with the limited-with > to only see the limited view. I don't see that. Each compilation has its own symbol table, and I don't see any reason why the mere fact of withing something that saw a limited view of some package was anything to do with a package that sees the full view. > You mentioned in another note that you have separate symbol table > entries for various different forms of a type. No, "types" aren't in the symbol table at all. They have their own separate table. Type names are in the symbol table, of course, as are component names. But a type name has nothing to do with a type -- they're completely separate concepts (as they are in Ada). > I think you will > need separate symbol table entries for the limited view and > the full view. In other words, for a library package P, you really have > two units, one whose library-level name is "P-full" and one whose name > is "P-limited". If that's true, we're getting into horrific complexity territory. A fundamental basis of the symbol table is that most entities have only one name and cannot be overloaded. You're asking that all of the lookup code be changed to be able to handle packages with two views. Along with all of the declaration code (so it can write into the correct view). Moreover, that would be true in every compiler (source based or library based). If the proposal requires both copies in the symboltable, then I think the proposal should be killed as soon as possible. If not sooner. ************************************************************** From: Pascal Leroy Sent: Wednesday, February 5, 2003 3:40 AM > I think you need to keep around indefinitely both a "full view" of > the package and a "limited view" of the package. Even after you > do a "full" compile, some units can still do "limited with" and should only > see the limited view. What I would recommend is you treat them as > pretty much distinct units. In terms of language description, you're right, a unit that does a "limited with" must see the limited view even for a unit that has been fully compiled. However, the implementation you suggest might make sense for your compiler, it doesn't make sense for ours (or for Randy's if I understand him right). We fundamentally depend on the invariant: 1 Ada unit = 1 Diana tree. Changing this is not an option. This means that if we see "limited with P" and P has been fully compiled, then we need to simulate/synthesize a limited view for P. For name resolution, this is simple enough. When we see the name P.T we do a lookup of the string "T" in the identifier table for P. The lookup has to succeed if and only if T is part of the limited view. We would probably do that by examining the Diana tree for T. Another option would be to store a bit "yes, I am part of the limited view" on the defining identifier for T. Once name resolution has been done, we will have access to the full tree for T. At this point we will need to behave as if T was incomplete. I can think of at least 3 ways to achieve this in our compiler, but the simplest one is probably to use a predicate "is this an incomplete type?" where appropriate, and have that predicate determine whether visibility was obtained though a "limited with". We already have such a predicate, but I'm pretty sure that we don't call it everywhere it would be needed. This certainly looks like work, but we are not talking man-years here. And there is certainly no need to keep two trees for each unit. ************************************************************** From: Jean-Pierre Rosen Sent: Wednesday, February 5, 2003 3:32 AM > Add a legality rule: > > A library_item mentioned in a limited_with_clause shall be a > package_declaration[, not a subprogram_declaration, generic_declaration, > or generic_instantiation]. Why not generic_instantiation? 1) not really different from a package declaration 2) quite useful, since instantiations are commonly used for building objects with multiple facets. Later you say: >We do not allow a limited_with_clause to mention a generic >instantiation, because that would require all kinds of semantic analysis >stuff, such as visibility. Well, the instantiation has been compiled at that point, so all visibility should be solved. Maybe worth some further investigation ************************************************************** From: Pascal Leroy Sent: Wednesday, February 5, 2003 3:59 AM The restriction is perfectly sensible. We don't want to have to do any name resolution to build the limited view, and in order to determine what generic we are talking about, we would have to do a pretty extensive name resolution (use clauses, parent units, renamings, etc.). Moreover, in order to build the instantiation, we would have to have compiled the specification of the generic already, but limited views cannot have compilation prerequisites (if they had, we could run into circularity problems). ************************************************************** From: Robert A. Duff Sent: Wednesday, February 5, 2003 8:22 AM Tucker and I discussed this issue yesterday, and we decided that although it *is* troubling to make rules that cause package instances to be different from normal packages, it is not feasible to implement the proposal (in some compilers) without this restriction. If there's a cycle, then the instantiation might *not* have been compiled yet. How about: limited with A; package B is new Some_Generic(...); limited with B; package A is new Some_Generic(...); It cannot be the case that A and B have both been *fully* compiled before each other. I really think a key feature of this proposal is that when the compiler sees "limited with X", it can determine the list of incomplete types in X in a purely syntactic manner. It should not have to do any kind of heavy-duty semantic analysis. It should not have to look at any source text outside of X. It's not *so* bad, because if you wanted to do cycles like the above, you can always break the cycle by adding more generic formal parameters. ************************************************************** From: Jean-Pierre Rosen Sent: Wednesday, February 5, 2003 11:28 AM Yes. I see the problem, and I have sympathy for the poor compiler writers. But with my teacher's hat on, I don't feel very easy to explain that in some cases, an instantiation is not equivalent to a regular package. ************************************************************** From: Tucker Taft Sent: Wednesday, February 5, 2003 5:47 AM Pascal Leroy wrote: >... > > From a pedagogical standpoint it would seem simpler to say that "limited > with" works like "with", i.e. that "limited with P.C.D" is equivalent to > "with P, P.C, P.C.D". Any other option is going to unnecessary confuse > users. > > So maybe the right answer is that if a package has a "limited with" of > itself it has no effect (but is not illegal). Yes, that is preferable. > Yes, I was thinking that use clauses and package renamings would be OK in > that context, although if they lead to problems we could disallow them. But > I can't think of any problem at this point. I sent a note about this, titled "Restrictions on limited-with". Have you seen it? Any comments? I think "use" clauses are a bad idea. Renaming is tricky, and needs careful thought. ************************************************************** From: Pascal Leroy Sent: Wednesday, February 5, 2003 6:23 AM Yes, I saw it, but after sending the above message. I agree that use clauses cause a Beaujolais-ish effect (maybe this should be named the Chianti effect ;-) and should be disallowed. I also agree that renamings should probably be disallowed since they cause more trouble than they are worth. ************************************************************** From: Robert Dewar Sent: Wednesday, February 5, 2003 7:18 AM So far it seems to me that the limited-with discussion looks like it is very promising. I must say I was unhappy with the idea of furiously trying to get a resolution in Padua (and I was trying to figure out how to molest my schedule to attend :-) but now I am actually getting some confidence that a) this is going in the right direction b) there really is a good possibility of agreement Very encouraging :-) We will definitely try to prototype this in GNAT as soon as there is a reasonably well defined proposal. ************************************************************** From: Tucker Taft Sent: Tuesday, February 11, 2003 2:21 PM Here is some initial analysis of implementation issues associated with "limited with." I tried to make most of the analysis technology independent, but of course that is nearly impossible, so it may apply less or more to technologies other than AdaMagic. -Tuck ----------------- Thoughts on the implementation of "limited with" in AdaMagic $Revision: 1.9 $ $Date: 2003/09/30 02:01:17 $ The "limited with" clause makes a "limited view" of a package visible in the compilation unit that has the limited-with clause, as well as all of its descendants. However, the limited view is hidden from all visibility (as are all the incomplete types and nested package limited views within it) under certain circumstances. After much discussion at the Padua ARG meeting, the best rule seemed to be that if the full view of the package is visible, including via one or more renamings, then the limited view is hidden from all visibility. [This was considerd necessary so as to avoid having two different views of the same package usable at the same point in the source text, which was felt to create both implementation complexity and possible user confusion.] SEPARATE "LIMITED" COMPILATION UNIT Our model for implementing a limited view of a library package will be that it has its "own" compilation-unit name, constructed from its "real" name but with suffix "'Limited". Hence P.Q.R'Limited is the limited view of P.Q.R. [Note that "P.Q.R.Limited" could also be used as the name, since "limited" is a reserved word, and would never be the real name of a package. However, it is very important that in any case P.Q.R.Limited or P.Q.R'Limited not depend semantically on P.Q.R itself.] In the representation of the library, it is important that P.Q.R'Limited can exist before P.Q.R has been fully compiled, and can retain some continuing existence while P.Q.R is being fully compiled, and thereafter. It is also important to realize that the packages nested within P.Q.R'Limited are themselves limited views. Hence, P.Q.R'Limited.NP always denotes a limited view of a package, if P.Q.R.NP denotes a nested package spec. Similarly, P.Q.R'Limited.T denotes an incomplete type declaration for T, if P.Q.R.T denotes a full type declaration. SEPARATE TIMESTAMP FOR LIMITED VIEW P.Q.R'Limited has its own timestamp, for the purposes of out-of-dateness checking. It is important that this *not* change when a "full" compile of P.Q.R is started, because that would make the units that P.Q.R depends on that themselves depend on P.Q.R'Limited, automatically appear out of date. Once the full compile of P.Q.R is complete, a check for consistency between P.Q.R'Limited and P.Q.R should be performed. If the views are no longer consistent, due to some change in P.Q.R, a new version of P.Q.R'Limited should be constructed, and its timestamp changed. This will effectively put all the dependents of P.Q.R'Limited out of date, and may in fact put P.Q.R itself out of date. If so, then P.Q.R will have to be compiled again. Ideally, this will be performed immediately, since the source code for P.Q.R is readily available at that time. However, depending on the compilation mechanism, it may be first necessary to compile other dependents of P.Q.R'Limited, so they are back in an up-to-date state before re-attempting the full compilation of P.Q.R. This might be left until link time, or to a separate "make" tool. Of course if P.Q.R'Limited is initially created by simply parsing the source text for P.Q.R, the timestamp of P.Q.R'Limited would be set at parse time. For those compilation systems that retain the date of modification of the source file as the time stamp of the corresponding units in the program library, some of this becomes simpler. Nevertheless, it seems a helpful, and in some cases necessary, "optimization" to *not* change the timestamp of the limited view every time the source text timestamp changes. Because the check for consistency between a full view and a limited view should be relatively easy to perform, there seems no need to change the timestamp on the limited view so long as it remains consistent with the full view, even if the full view corresponds to source text with a different timestamp. This means that units that depend only on the P.Q.R'Limited are isolated from "minor" changes to P.Q.R, and are only affected when a type or nested package is added or removed from P.Q.R. In other words, you get a "poor man's" version of incremental compilation as a side effect of implementing limited views of packages. HIDING THE LIMITED VIEW As mentioned above, a limited view of a package can be hidden from all visibility. The anticipated rule is that the limited view is hidden from all visibility if the full view is visible either "normally" or via a rename. This implies that if P.Q.R is visible, including via a rename, then P.Q.R'Limited is not visible. Furthermore, even when P.Q.R'Limited is visible, it is possible that P.Q.R'Limited.NP might not be visible, because P.Q.R.NP might be visible via a renaming. A relatively simple implementation of this requirement would seem to be to build up a set representing those (external) packages that have a full view visible, including via a rename, within the current compilation unit. (By "external" we mean it is from some other compilation unit.) This set will not grow once the context clause has been fully processed, since it is not possible to locally declare a renaming of a package that is not already visible. This set of visible packages would probably be represented using a hash table of some sort. It would start out empty when starting to compile a new compilation unit, and would grow each time a (real, not limited) "with" for a package or a package renaming is processed. Each of these "with"ed packages would be scanned for enclosed renamings, recursively. [Alternatively, the representation of every package could include a set of all enclosed renamings (directly or indirectly).] The "with"ed package, plus all of the packages for which it contains a renaming, would be unioned into this set. Later, whenever a limited view of a package is encountered within a name, a check is made to see whether a corresponding full view exists in this set. If so, the limited view may not be named, and so the name is illegal. Another anticipated rule is that a limited with clause is illegal if the package is already visible, including via a rename. This would seem to be friendlier than allowing the programmer to put a "limited with" on a compilation unit and then not be able to use it at all. However, there will still be "limited with"s inherited from ancestor units, and these will be essentially ignored if the corresponding package is fully visible some other way. Also, in a limited with clause like "limited with P.Q.R;" presumably implicit limited withs for P and P.Q are provided, but such an implicit limtied with again must be ignored if the package mentioned is already fully visible via some other means. To enforce the above rule regarding disallowing and/or ignoring limited withs, it may be necessary to do a second pass over the context clause, after the full set of visible packages has been built up. In many cases, a second pass is already necessary in Ada 95 to correctly deal with "with"s of private units, since it is not known whether those with's are legal until the name of the compilation unit being compiled is known, so it may be possible to combine these error-checking passes. WHEN IS THE INCOMPLETE TYPE COMPLETE? Our current plan is to disallow "use" clauses and renamings for limited views. Given that, the only way to name one of the incomplete types inside a limited view is by naming the enclosing package, so there is no need to have a separate mechanism for determining whether the incomplete type declaration is visible. It is visible exactly when the enclosing limited view can be named. However, there are access types that might have been declared using the incomplete type, and it is important to know when a dereference of the access type is considered "complete." The basic rule would be that when the full type is visible, including via a rename of the enclosing package, (or equivalently, when the incomplete type declaration is hidden from all visibility), the access type may be dereferenced to produce the full type. This implies that any time such an access type is dereferenced, a check should be made whether the designated type's enclosing limited view(s) are all still visible. If they are all still visible, then the incomplete type is still visible, and the dereference produces an incomplete object. If any one of them is hidden from all visibility, then the full type is visible, and the dereference produces an object of a "full" type. Some optimization of this check is probably possible. Even when a dereference of the access type is considered incomplete, it may still be legal in a context where a full type would normally be required, so long as the expected type is the full type, or the full type is "nearby." One of the versions of the type-stub proposal laid out the rules for this matching between incomplete and full type, and those rules seem still relevant for all of the other proposals as well. Here is a copy of those rules (from AI-00217-04/04): A dereference (implicit or explicit) of a value of an access type whose designated type D is incomplete is allowed only in the following contexts: * in a place where the completion of D is available (see above); * in a context where the expected type is E and o E covers the completion of D, o E is tagged and covers D, o E covers D'Class or its completion, or o E'Class covers D or its completion; * as the target of an assignment_statement where the type of the value being assigned is V, and V or V'Class is the completion of D. In these contexts, the incomplete type is defined to be the same type as its completion, and its first subtype statically matches the first subtype of its completion. The first bullet would be replaced by "where the full type declaration for D is visible" for the limited-with proposal (and for the "type C.T;" proposal). ************************************************************** From: Robert A. Duff Sent: Tuesday, February 11, 2003 5:34 PM ... > SEPARATE "LIMITED" COMPILATION UNIT > > Our model for implementing a limited view of a library package > will be that it has its "own" compilation-unit name, > constructed from its "real" name but with suffix "'Limited". I don't see why you want it to have a different name. It seems to me that there are two declarations of views, both called P.Q.R, and the full view hides the limited view. > Hence P.Q.R'Limited is the limited view of P.Q.R. > [Note that "P.Q.R.Limited" could also be used as the > name, since "limited" is a reserved word, and would > never be the real name of a package. However, it is > very important that in any case P.Q.R.Limited or > P.Q.R'Limited not depend semantically on P.Q.R itself.] Right -- the dependence is the other way 'round. > In the representation of the library, it is important > that P.Q.R'Limited can exist before P.Q.R has been > fully compiled, and can retain some continuing existence > while P.Q.R is being fully compiled, and thereafter. > It is also important to realize that the packages > nested within P.Q.R'Limited are themselves limited > views. And *they* don't get weird names... >...Hence, P.Q.R'Limited.NP always denotes a limited view > of a package, if P.Q.R.NP denotes a nested package spec. Similarly, > P.Q.R'Limited.T denotes an incomplete type declaration for T, > if P.Q.R.T denotes a full type declaration. > > SEPARATE TIMESTAMP FOR LIMITED VIEW > > P.Q.R'Limited has its own timestamp, for the purposes > of out-of-dateness checking. It is important that this > *not* change when a "full" compile of P.Q.R is started, > because that would make the units that P.Q.R depends > on that themselves depend on P.Q.R'Limited, automatically > appear out of date. Right. It seems to me that in a source-based library model, the timestamp of the limited view can be the timestamp of the source file containing that package, whereas the timestamp of the full view is really a collection of timestamps for all the source files the package depends upon. > Once the full compile of P.Q.R is complete, a check for > consistency between P.Q.R'Limited and P.Q.R should be > performed. I don't understand the need for this consistency check. It seems to me that P.Q.R depends upon P.Q.R'Limited, and you do the normal check at link time, to make sure that two different versions of P.Q.R'Limited are not included in the same partition. In normal operation, you just do a "build" command, and the only way the link-time check can fail is if you edit your source code while the build is running. >... If the views are no longer consistent, due to > some change in P.Q.R, a new version of P.Q.R'Limited should > be constructed, and its timestamp changed. This will effectively > put all the dependents of P.Q.R'Limited out of date, > and may in fact put P.Q.R itself out of date. If so, > then P.Q.R will have to be compiled again. Ideally, this > will be performed immediately, since the source code for > P.Q.R is readily available at that time. However, depending on the > compilation mechanism, it may be first necessary to compile > other dependents of P.Q.R'Limited, so they are back in > an up-to-date state before re-attempting the full > compilation of P.Q.R. This might be left until link time, > or to a separate "make" tool. I don't see the need for such complexity in the "make" tool (recompiling things twice and whatnot). It seems no different than the current language: if you edit package Foo in between compiling two different things that depend on Foo, then you get a link-time failure, and run the "make" tool again. > Of course if P.Q.R'Limited is initially created by simply parsing > the source text for P.Q.R, the timestamp of P.Q.R'Limited would be > set at parse time. > > For those compilation systems that retain the date of modification > of the source file as the time stamp of the corresponding units > in the program library, some of this becomes simpler. > Nevertheless, it seems a helpful, and in some cases > necessary, "optimization" to *not* change the timestamp of > the limited view every time the source text timestamp changes. > Because the check for consistency between a full view and > a limited view should be relatively easy to perform, there > seems no need to change the timestamp on the limited view > so long as it remains consistent with the full view, even > if the full view corresponds to source text with a different > timestamp. This means that units that depend only on the > P.Q.R'Limited are isolated from "minor" changes to P.Q.R, > and are only affected when a type or nested package is > added or removed from P.Q.R. In other words, you get > a "poor man's" version of incremental compilation as a side > effect of implementing limited views of packages. Seems like a nice optimization, but I don't see why it's necessary. > HIDING THE LIMITED VIEW > > As mentioned above, a limited view of a package can be hidden from > all visibility. The anticipated rule is that the limited > view is hidden from all visibility if the full view is visible > either "normally" or via a rename. This implies that if > P.Q.R is visible, including via a rename, then P.Q.R'Limited > is not visible. Furthermore, even when P.Q.R'Limited > is visible, it is possible that P.Q.R'Limited.NP might > not be visible, because P.Q.R.NP might be visible via > a renaming. > > A relatively simple implementation of this requirement > would seem to be to build up a set representing those (external) packages > that have a full view visible, including via a rename, within the > current compilation unit. (By "external" we mean it is from some > other compilation unit.) This set will not grow once the > context clause has been fully processed, since it is not > possible to locally declare a renaming of a package that is not > already visible. Well, sort of. Not just the context clause, but the parent unit name, and all the context clauses that get inherited by children and subunits and bodies. > This set of visible packages would probably be represented > using a hash table of some sort. It would start out empty > when starting to compile a new compilation unit, and would > grow each time a (real, not limited) "with" for a package or a package > renaming is processed. Each of these "with"ed packages > would be scanned for enclosed renamings, recursively. > [Alternatively, the representation of every package could > include a set of all enclosed renamings (directly or indirectly).] > The "with"ed package, plus all of the packages for which it > contains a renaming, would be unioned into this set. > > Later, whenever a limited view of a package is encountered > within a name, a check is made to see whether a corresponding > full view exists in this set. If so, the limited view may > not be named, and so the name is illegal. I don't undersstand this. First, it seems like this is all part of overload resolution: you see a name, and it will resolve to the full view, if the full view is visible. Second, why do you concentrate on names? I mean, you run across types in the symbol table all the time, and it seems like you need to make the decision there as well. (Consider the examples in my earlier e-mail, where you refered to a variable that is of the type, a subtype of the type, or whatever.) > Another anticipated rule is that a limited with clause is > illegal if the package is already visible, including > via a rename. This would seem to be friendlier than allowing > the programmer to put a "limited with" on a compilation unit and > then not be able to use it at all. Perhaps a little bit friendlier, but it's not the simplest rule, and you're not really protecting the user against any serious damage. >... However, there will still > be "limited with"s inherited from ancestor units, and these > will be essentially ignored if the corresponding package > is fully visible some other way. Also, in a limited with > clause like "limited with P.Q.R;" presumably implicit > limited withs for P and P.Q are provided, but such an implicit > limtied with again must be ignored if the package mentioned > is already fully visible via some other means. > > To enforce the above rule regarding disallowing and/or ignoring > limited withs, it may be necessary to do a second pass over > the context clause, after the full set of visible packages has > been built up. In many cases, a second pass is already necessary > in Ada 95 to correctly deal with "with"s of private units, since it is > not known whether those with's are legal until the name of the > compilation unit being compiled is known, so it may be possible > to combine these error-checking passes. The existing rule requires looking at the comp unit name before processing the context clause. It doesn't require two passes over the context clause. (Either way, it's not the "one pass semantics" that you and Robert Dewar like so much. ;-) ) > WHEN IS THE INCOMPLETE TYPE COMPLETE? > > Our current plan is to disallow "use" clauses and renamings > for limited views. Given that, the only way to name one > of the incomplete types inside a limited view is by naming > the enclosing package, so there is no need to have a separate mechanism > for determining whether the incomplete type declaration is visible. > It is visible exactly when the enclosing limited view can be named. > > However, there are access types that might have been declared > using the incomplete type, and it is important to know when a > dereference of the access type is considered "complete." And parameters (in the tagged case). Normally, the body will 'with' the full view, so it can do stuff with the type. But if you forget that 'with', the compiler needs to notice that the type is still incomplete. > The basic rule would be that when the full type is visible, > including via a rename of the enclosing package, (or equivalently, > when the incomplete type declaration is hidden from all visibility), > the access type may be dereferenced to produce the full type. > This implies that any time such an access type is dereferenced, > a check should be made whether the designated type's enclosing limited > view(s) are all still visible. If they are all still visible, > then the incomplete type is still visible, and the dereference > produces an incomplete object. I don't think you mean, "produces an incomplete object". I think you mean, simply, "is illegal". I'm not sure I understand the "all" here. I mean, just because the full view of *one* such containing package is visible doesn't make the full view of the type visible, does it? >...If any one of them is hidden from > all visibility, then the full type is visible, and the dereference > produces an object of a "full" type. Some optimization of this > check is probably possible. > > Even when a dereference of the access type is considered > incomplete, it may still be legal in a context where a full > type would normally be required, so long as the expected type > is the full type, or the full type is "nearby." One of the > versions of the type-stub proposal laid out the rules for > this matching between incomplete and full type, and those > rules seem still relevant for all of the other proposals as well. Oh, I see. So what I said above is wrong. Hmm. > Here is a copy of those rules (from AI-00217-04/04): > > A dereference (implicit or explicit) of a value of an access type whose > designated type D is incomplete is allowed only in the following > contexts: > > * in a place where the completion of D is available (see above); > > * in a context where the expected type is E and > o E covers the completion of D, > o E is tagged and covers D, > o E covers D'Class or its completion, or > o E'Class covers D or its completion; > > * as the target of an assignment_statement where the type of the value > being assigned is V, and V or V'Class is the completion of D. How can V'Class be the completion of something? > In these contexts, the incomplete type is defined to be the same > type as its completion, and its first subtype statically matches the > first subtype of its completion. > > The first bullet would be replaced by "where the full type declaration > for D is visible" for the limited-with proposal (and for the > "type C.T;" proposal). ************************************************************** From: Tucker Taft Sent: Tuesday, February 11, 2003 10:28 PM Robert A Duff wrote: >>SEPARATE "LIMITED" COMPILATION UNIT >> >>Our model for implementing a limited view of a library package >>will be that it has its "own" compilation-unit name, >>constructed from its "real" name but with suffix "'Limited". > > > I don't see why you want it to have a different name. It seems to me > that there are two declarations of views, both called P.Q.R, and the > full view hides the limited view. Some of this is in reaction to Randy and Pascal's comments on my earlier implementation musings. They don't like the idea of having two units in the library with the same name. It breaks their model badly. So I was proposing that they have different names, at least as far as the program library mechanism is concerned. In general, the program library mechanism is presumed to be too stupid to understand hiding. And the limited view might be needed even when the full view has been added to the program library, because a unit explicitly requests the limited view via "limited with." >>Hence P.Q.R'Limited is the limited view of P.Q.R. >>[Note that "P.Q.R.Limited" could also be used as the >>name, since "limited" is a reserved word, and would >>never be the real name of a package. However, it is >>very important that in any case P.Q.R.Limited or >>P.Q.R'Limited not depend semantically on P.Q.R itself.] > > > Right -- the dependence is the other way 'round. If there is any dependence at all. They both depend on the source text for P.Q.R, but only the full view depends on the source text for other units on which it is semantically dependent. >>In the representation of the library, it is important >>that P.Q.R'Limited can exist before P.Q.R has been >>fully compiled, and can retain some continuing existence >>while P.Q.R is being fully compiled, and thereafter. >>It is also important to realize that the packages >>nested within P.Q.R'Limited are themselves limited >>views. > > > And *they* don't get weird names... Right. The "weird" names are just so the program library mechanism can be relatively stupid, and just deliver up one and only one unit given a particular name. >>...Hence, P.Q.R'Limited.NP always denotes a limited view >>of a package, if P.Q.R.NP denotes a nested package spec. Similarly, >>P.Q.R'Limited.T denotes an incomplete type declaration for T, >>if P.Q.R.T denotes a full type declaration. >> >>SEPARATE TIMESTAMP FOR LIMITED VIEW >> >>P.Q.R'Limited has its own timestamp, for the purposes >>of out-of-dateness checking. It is important that this >>*not* change when a "full" compile of P.Q.R is started, >>because that would make the units that P.Q.R depends >>on that themselves depend on P.Q.R'Limited, automatically >>appear out of date. > > > Right. It seems to me that in a source-based library model, the > timestamp of the limited view can be the timestamp of the source file > containing that package, whereas the timestamp of the full view is > really a collection of timestamps for all the source files the package > depends upon. I was trying to handle various approaches to time stamps. In our model, each compiled representation keeps the timestamps of all the source files it depends on. As you imply above, P.Q.R'Limited would depend on exactly one source file ("p.q.r.spc", for example), whereas P.Q.R would probably depend on many source files, including p.q.r.spc as well. I believe in Randy's model, there is a timestamp associated with the compiled representation itself, and it must be >= the timestamps of all other compiled representations it depends on. I don't believe he keeps source file timestamps at all. Timestamps are associated with when a unit is submitted to the compiler, I believe. I think Rational has aspects of both timestamp models, though I will let Pascal talk about the details. >>Once the full compile of P.Q.R is complete, a check for >>consistency between P.Q.R'Limited and P.Q.R should be >>performed. > > I don't understand the need for this consistency check. > It seems to me that P.Q.R depends upon P.Q.R'Limited, > and you do the normal check at link time, to make sure that > two different versions of P.Q.R'Limited are not included in the same > partition. > > In normal operation, you just do a "build" command, and the only way the > link-time check can fail is if you edit your source code while the build > is running. Again, I was trying to accommodate a model where timestamps are associated with compiled representations, rather than only with source text. I agree that a purely source-based model can be simpler. >>... If the views are no longer consistent, due to >>some change in P.Q.R, a new version of P.Q.R'Limited should >>be constructed, and its timestamp changed. This will effectively >>put all the dependents of P.Q.R'Limited out of date, >>and may in fact put P.Q.R itself out of date. If so, >>then P.Q.R will have to be compiled again. Ideally, this >>will be performed immediately, since the source code for >>P.Q.R is readily available at that time. However, depending on the >>compilation mechanism, it may be first necessary to compile >>other dependents of P.Q.R'Limited, so they are back in >>an up-to-date state before re-attempting the full >>compilation of P.Q.R. This might be left until link time, >>or to a separate "make" tool. > > > I don't see the need for such complexity in the "make" tool > (recompiling things twice and whatnot). It seems no different > than the current language: if you edit package Foo in between compiling > two different things that depend on Foo, then you get a link-time > failure, and run the "make" tool again. Yes, I agree the source-based model can be simpler. However, Randy seemed to find the timestamp issue a big problem with "limited with," so I was making some attempt to address that. I probably shouldn't have strayed away from the source-based model. >>Of course if P.Q.R'Limited is initially created by simply parsing >>the source text for P.Q.R, the timestamp of P.Q.R'Limited would be >>set at parse time. >> >>For those compilation systems that retain the date of modification >>of the source file as the time stamp of the corresponding units >>in the program library, some of this becomes simpler. >>Nevertheless, it seems a helpful, and in some cases >>necessary, "optimization" to *not* change the timestamp of >>the limited view every time the source text timestamp changes. >>Because the check for consistency between a full view and >>a limited view should be relatively easy to perform, there >>seems no need to change the timestamp on the limited view >>so long as it remains consistent with the full view, even >>if the full view corresponds to source text with a different >>timestamp. This means that units that depend only on the >>P.Q.R'Limited are isolated from "minor" changes to P.Q.R, >>and are only affected when a type or nested package is >>added or removed from P.Q.R. In other words, you get >>a "poor man's" version of incremental compilation as a side >>effect of implementing limited views of packages. > > > Seems like a nice optimization, but I don't see why it's necessary. I think it might be necessary in the non-source-based model, if timestamps are associated with the time when units are submitted to the compiler, rather than with when the source text is edited. > > >>HIDING THE LIMITED VIEW >> >>As mentioned above, a limited view of a package can be hidden from >>all visibility. The anticipated rule is that the limited >>view is hidden from all visibility if the full view is visible >>either "normally" or via a rename. This implies that if >>P.Q.R is visible, including via a rename, then P.Q.R'Limited >>is not visible. Furthermore, even when P.Q.R'Limited >>is visible, it is possible that P.Q.R'Limited.NP might >>not be visible, because P.Q.R.NP might be visible via >>a renaming. >> >>A relatively simple implementation of this requirement >>would seem to be to build up a set representing those (external) packages >>that have a full view visible, including via a rename, within the >>current compilation unit. (By "external" we mean it is from some >>other compilation unit.) This set will not grow once the >>context clause has been fully processed, since it is not >>possible to locally declare a renaming of a package that is not >>already visible. > > > Well, sort of. Not just the context clause, but the parent unit name, > and all the context clauses that get inherited by children and subunits > and bodies. Right. > > >>This set of visible packages would probably be represented >>using a hash table of some sort. It would start out empty >>when starting to compile a new compilation unit, and would >>grow each time a (real, not limited) "with" for a package or a package >>renaming is processed. Each of these "with"ed packages >>would be scanned for enclosed renamings, recursively. >>[Alternatively, the representation of every package could >>include a set of all enclosed renamings (directly or indirectly).] >>The "with"ed package, plus all of the packages for which it >>contains a renaming, would be unioned into this set. >> >>Later, whenever a limited view of a package is encountered >>within a name, a check is made to see whether a corresponding >>full view exists in this set. If so, the limited view may >>not be named, and so the name is illegal. > > > I don't undersstand this. First, it seems like this is all part of > overload resolution: you see a name, and it will resolve to the full > view, if the full view is visible. The problem is package renames. We concluded that if there is a renaming of P.Q.R visible, then the limited view of P.Q.R should become invisible, even though "P.Q.R" does not denote the full view. So this is a case where a name is hidden from all visibility, even though it isn't a homograph with the thing doing the hiding. This is somewhat like a more extreme version of the situation where having one overloadable in a use-visible set hides all the non-overloadables in the use-visible set, and vice versa. Here, S.Renaming_Of_P_Q_R is hiding P.Q.R'Limited from all visibility. > > Second, why do you concentrate on names? I mean, you run across types > in the symbol table all the time, and it seems like you need to make the > decision there as well. Well, I talk about that later. If the types are already in the symbol table, the question is whether they should be considered incomplete or complete. The name issue is whether the name is even visible. > ... > (Consider the examples in my earlier e-mail, > where you refered to a variable that is of the type, a subtype of the > type, or whatever.) I gave an adequate explanation of that there, hopefully. > > >>Another anticipated rule is that a limited with clause is >>illegal if the package is already visible, including >>via a rename. This would seem to be friendlier than allowing >>the programmer to put a "limited with" on a compilation unit and >>then not be able to use it at all. > > > Perhaps a little bit friendlier, but it's not the simplest rule, > and you're not really protecting the user against any serious damage. That's true. But it would seem mysterious if the reason the limited with has no effect is because there is a renaming in some "with"ed package. I think the programmer should be told about that as soon as possible, and the limited with clause seems like the ideal place. > > >>... However, there will still >>be "limited with"s inherited from ancestor units, and these >>will be essentially ignored if the corresponding package >>is fully visible some other way. Also, in a limited with >>clause like "limited with P.Q.R;" presumably implicit >>limited withs for P and P.Q are provided, but such an implicit >>limtied with again must be ignored if the package mentioned >>is already fully visible via some other means. >> >>To enforce the above rule regarding disallowing and/or ignoring >>limited withs, it may be necessary to do a second pass over >>the context clause, after the full set of visible packages has >>been built up. In many cases, a second pass is already necessary >>in Ada 95 to correctly deal with "with"s of private units, since it is >>not known whether those with's are legal until the name of the >>compilation unit being compiled is known, so it may be possible >>to combine these error-checking passes. > > > The existing rule requires looking at the comp unit name before > processing the context clause. It doesn't require two passes over the > context clause. (Either way, it's not the "one pass semantics" that you > and Robert Dewar like so much. ;-) ) You can do it either way. AdaMagic doesn't peak ahead at the comp-unit name. Instead, it processes all the "with" clauses and then when it reaches the comp-unit name, it checks to be sure they are legal. So if you do it that way, you already have that second pass over the context clause. >>WHEN IS THE INCOMPLETE TYPE COMPLETE? >> >>Our current plan is to disallow "use" clauses and renamings >>for limited views. Given that, the only way to name one >>of the incomplete types inside a limited view is by naming >>the enclosing package, so there is no need to have a separate mechanism >>for determining whether the incomplete type declaration is visible. >>It is visible exactly when the enclosing limited view can be named. >> >>However, there are access types that might have been declared >>using the incomplete type, and it is important to know when a >>dereference of the access type is considered "complete." > > > And parameters (in the tagged case). Normally, the body will 'with' the > full view, so it can do stuff with the type. But if you forget that > 'with', the compiler needs to notice that the type is still incomplete. Right. >>The basic rule would be that when the full type is visible, >>including via a rename of the enclosing package, (or equivalently, >>when the incomplete type declaration is hidden from all visibility), >>the access type may be dereferenced to produce the full type. >>This implies that any time such an access type is dereferenced, >>a check should be made whether the designated type's enclosing limited >>view(s) are all still visible. If they are all still visible, >>then the incomplete type is still visible, and the dereference >>produces an incomplete object. > > > I don't think you mean, "produces an incomplete object". I think you > mean, simply, "is illegal". That used to be true. But with incomplete tagged types, it is legal to dereference under certain circumstances. Also, the "extra" matching rules also imply that the dereference can be legal, even for a non-tagged incomplete type. > I'm not sure I understand the "all" here. I mean, just because the full > view of *one* such containing package is visible doesn't make the full > view of the type visible, does it? Yes, if the full view of any containing package is visible, then everything inside the package is definitely visible. By "containing" I should say "textually" containing. I don't mean to include ancestor library packages. >>...If any one of them is hidden from >>all visibility, then the full type is visible, and the dereference >>produces an object of a "full" type. Some optimization of this >>check is probably possible. >> >>Even when a dereference of the access type is considered >>incomplete, it may still be legal in a context where a full >>type would normally be required, so long as the expected type >>is the full type, or the full type is "nearby." One of the >>versions of the type-stub proposal laid out the rules for >>this matching between incomplete and full type, and those >>rules seem still relevant for all of the other proposals as well. > > > Oh, I see. So what I said above is wrong. Hmm. > > >>Here is a copy of those rules (from AI-00217-04/04): The second and third bullets below are what I have been referring to as the "extra" matching rules. >> >> A dereference (implicit or explicit) of a value of an access type whose >> designated type D is incomplete is allowed only in the following >> contexts: >> >> * in a place where the completion of D is available (see above); >> >> * in a context where the expected type is E and >> o E covers the completion of D, >> o E is tagged and covers D, >> o E covers D'Class or its completion, or >> o E'Class covers D or its completion; >> >> * as the target of an assignment_statement where the type of the value >> being assigned is V, and V or V'Class is the completion of D. > > > How can V'Class be the completion of something? The "full" implicitly declared type P.Q.R.V'Class can be the completion of the (implicitly declared) incomplete type P.Q.R'Limited.V'Class. ************************************************************** From: Robert A. Duff Sent: Wednesday, February 12, 2003 2:40 PM Tucker says: > Some of this is in reaction to Randy and Pascal's > comments on my earlier implementation musings. > They don't like the idea of having two units in the library with > the same name. It breaks their model badly. So > I was proposing that they have different names, > at least as far as the program library mechanism is > concerned. It seems just like spec vs. body. Now we have limited view vs. spec. > > I don't understand the need for this consistency check. > > It seems to me that P.Q.R depends upon P.Q.R'Limited, > > and you do the normal check at link time, to make sure that > > two different versions of P.Q.R'Limited are not included in the same > > partition. > > > > In normal operation, you just do a "build" command, and the only way the > > link-time check can fail is if you edit your source code while the build > > is running. > > Again, I was trying to accommodate a model where timestamps > are associated with compiled representations, rather than > only with source text. I agree that a purely source-based > model can be simpler. I don't see what difference the library model makes in this regard. If you have a "build" command, it has to look at timestamps of source files. It doesn't matter whether those time stamps are recorded with the compilation artifacts, or whether those get the timestamp of when they were generated. Either way, so long as the user is not doing something in parallel (like editing the source, or deleting the compilation artifacts by hand), then end result will be a consistent build result. In other words, the job of a "build" tool is to bring everything up to date, whatever that means for the library model in use. If the build tool complains about things being out of date, then it's broken (except when the user is meddling in parallel as described above). I claim this is possible to achieve in *any* of the models, without extra complexity (such as when you talked about compile the same thing twice). Of course, implementations are not required to support "build" tools, and if they don't then the user has to worry about more stuff. But we were talking about what a build tool would do... Perhaps Randy or Pascal can explain about non-source-based compilers better.... ************************************************************** From: Tucker Taft Sent: Wednesday, February 12, 2003 3:33 PM Robert A Duff wrote: > It seems just like spec vs. body. Now we have limited view vs. spec. Yes, that is similar. The "full" name of the unit in the program library could really be "P.Q.R", kind => spec/body/limited-view>. > > > I don't understand the need for this consistency check. > > > It seems to me that P.Q.R depends upon P.Q.R'Limited, > > > and you do the normal check at link time, to make sure that > > > two different versions of P.Q.R'Limited are not included in the same > > > partition. > > > > > > In normal operation, you just do a "build" command, and the only way the > > > link-time check can fail is if you edit your source code while the build > > > is running. > > > > Again, I was trying to accommodate a model where timestamps > > are associated with compiled representations, rather than > > only with source text. I agree that a purely source-based > > model can be simpler. > > I don't see what difference the library model makes in this regard. If > you have a "build" command, it has to look at timestamps of source > files. It doesn't matter whether those time stamps are recorded with > the compilation artifacts, or whether those get the timestamp of when > they were generated. I'll let Randy explain whatever problem he was foreseeing. I may not have it right in any case. If I remember correctly, Randy implied that the source file timestamps were useless on Windows, because they weren't reliable or didn't have enough resolution, so his compiler generated sequential timestamps that were associated with the order in which files were submitted to the compiler. His build tool determines an order that should work, and perhaps queries the compiler to determine which units are considered out of date, and determines a sequence for just those units. > Either way, so long as the user is not doing something in parallel (like > editing the source, or deleting the compilation artifacts by hand), then > end result will be a consistent build result. > > In other words, the job of a "build" tool is to bring everything up to > date, whatever that means for the library model in use. If the build > tool complains about things being out of date, then it's broken (except > when the user is meddling in parallel as described above). I think this presumes a build tool that understands the out-of-dateness rules. I'm not sure that is how RR's works. > I claim this is possible to achieve in *any* of the models, without > extra complexity (such as when you talked about compile the same thing > twice). > > Of course, implementations are not required to support "build" tools, > and if they don't then the user has to worry about more stuff. > But we were talking about what a build tool would do... I wasn't talking about what a build tool would do, but maybe you were. I was talking about what the compiler would do when presented with a request to "fully" compile a unit, when it had earlier been parsed to produce a limited view. If the full view and the limited view share a single sequentially-assigned time stamp, then you can get yourself into a tangle. If the full view and the limited view have distinct time stamps, then things can work out. But you have no way of knowing whether the limited view and the full view were created from the same version of the source file if you don't believe source file timestamps. The only way would be to compare the results of the two different ways of processing the same file. I was suggesting that you compare the representation of the limited view with the representation of the full view, and if they are consistent, then there is no need to change the time stamp assigned to the limited view when it was first parsed. However, if they are inconsistent, then you need to recreate the limited view, and need to assign it the next sequential time stamp to make everything that depended on it out of date. > Perhaps Randy or Pascal can explain about non-source-based compilers > better.... I presume so as well. And I suspect that Rational and RR have different timestamp mechanisms. So I suppose what I really did was propose an implementation model for a library based on sequentially assigned time stamps, where the modification date of the source text is not saved, nor generally believed. Our compiler, on the other hand, bases everything on the modification date of the source text, and, as you point out, if you do that, the only way things can get out of sync is if the file is edited in the middle of a "build." ************************************************************** From: Robert A. Duff Sent: Wednesday, February 12, 2003 4:30 PM > If I remember correctly, Randy implied that > the source file timestamps were useless on Windows, because > they weren't reliable or didn't have enough resolution, I know of two problems with windows file timestamps, which have caused trouble for AdaMagic in the past: It does weird things with daylight-savings time, and with time zones, so that if you compiled everything in the summer, and then waited until winter (or vice versa) it would wrongly think everything it out of date and need recompiling, and similarly that shipping a program library to a different time zone would cause everything to seem out of date. Different windows file systems store the timestamps with different granularity (half second versus 1 second, or something like that), so that if you copy a directory from one to the other it would perturb some of the timestamps by, say, half a second. We found workarounds to both of these problems, but I sympathize with Randy: it was a pain. > so his compiler generated sequential timestamps that were > associated with the order in which files were submitted to > the compiler. ************************************************************** From: Robert Dewar Sent: Wednesday, February 12, 2003 10:27 PM > It does weird things with daylight-savings time, and with time zones, so > that if you compiled everything in the summer, and then waited until > winter (or vice versa) it would wrongly think everything it out of date > and need recompiling, and similarly that shipping a program library to a > different time zone would cause everything to seem out of date. This sounds like something that results from an improper attempt to use GMT time stamps on windows. That's always a mistake. ************************************************************** From: Randy Brukardt Sent: Thursday, February 13, 2003 2:09 PM > > It does weird things with daylight-savings time, and with time zones, so > > that if you compiled everything in the summer, and then waited until > > winter (or vice versa) it would wrongly think everything it out of date > > and need recompiling, and similarly that shipping a program library to a > > different time zone would cause everything to seem out of date. > > This sounds like something that results from an improper attempt to use > GMT time stamps on windows. That's always a mistake. I'm not sure. But I do know that when daylight saving time changes, all of the file times change an hour. It's not just GMT time stamps, if you look at files in Explorer, you can see this effect. (This is certainly true on Windows 2000, I don't recall seeing it on NT or 95.) I hadn't realized that it happened for timezones as well until Bob mentioned it, although that makes perfect sense. Most of the problems we had, however, was that the granularity of source changes is fairly large, and its possible to have multiple time stamps that are identical. In the worst cases (such as subunits updated by a tool), this potentially caused out-of-date things to fail to get recompiled. Anyway, the build tool uses the source timestamps anyway, but the compiler uses a timestamp for each generated symbol file created at the creation (or rewrite) of the file. We couldn't use the timestamps directly because of the various problems. We then compare those timestamps. (Those wouldn't really have to be timestamps at all; anything strictly accending without duplication would work). ************************************************************** From: Tucker Taft Sent: Tuesday, February 11, 2003 2:17 PM I thought I would update the full ARG on the Padua discussions relating to AI-00217, the mutual-dependence AI. This is rightly the job of the minute taker or Pascal, but I thought some of you might be waiting with bated breath... So this is a *very* unofficial summary of what we decided. Well, we *didn't* decide on a single solution. However, we did settle on an approach to make the decision, and we came to a number of decisions that limit the number of proposals that remain alive, and that answer some of the tricky semantic questions that all of the proposals are facing. We agreed to proceed with a full write up of the following three proposals: 1) type stubs, limited to a single stub per full type, where the package containing the full type must have a "with" for the package containing the stub, and where a "separate with ..." context clause is required if a stub appears inside a package. Randy will write this one up. 2) incomplete type to be completed in a child/nested package. (aka "type C.T;"). Tuck will write this one up. 3) "limited with" of a package. Pascal will write this one up, presumably starting with Bob's version. As usual, the writeups should include wording, rationale, pros and cons, various examples, known gotchas, etc. In addition, Randy, Tuck, and Pascal will produce at least one extended example of use using "their" proposal, and then exchange examples and do the same for the examples developed by the other two. Finally, all three will develop implementation models for all three proposals within their respective compiler technologies. [NOTE: Tuck has an initial draft for "limited with" in the "AdaMagic" technology, which will be sent out shortly.] Tuck will include a presentation of these three alternatives at the upcoming Ada UK meeting on April 8/9 in Swindon, and hope to get some useful user input. The extended examples should be available by this time for inclusion in the presentation. The implementation models need not be available by this date. During and after this sequence, the ARG will undergo its typically efficient process of e-mail analysis ;-), and no later than the end of the first day of the next ARG meeting, make a decision. ---------------- As far as specific technical decisions we made: 1) When the full type declaration is visible, including via a renaming of its enclosing package, the incomplete type declaration is hidden from all visibility (type C.T and limited-with proposals), or the type stub becomes equivalent to the full type (type stub proposal). 2) When a package is visible, including via a renaming, any "limited view" of the package is hidden from all visibility (type C.T and limited-with proposals). By "limited view" we mean the view of a package or nested package which is made visible by the limited-with, or the child/nested package used as a prefix in the "type C.T;" proposal. 3) A "limited with" would not be allowed if a full view of the package named is visible via some other means (limited with proposal). An inherited limited with, or an implicit limited with of a package mentioned (but not the specifically "with"ed package itself) in a limited with clause (e.g. "P.Q" in "limited with P.Q.R"), would be ignored if the full view of the package is visible via some other means. 4) No use clauses or renamings are permitted for a limited view of a package (type C.T and limited-with proposals). [Rationale: Use clauses seem to introduce possible Beaujolais-like effects, and renamings of limited views just make the "hidden from all visibility" rules even weirder.] 5) "Limited with" would be available only on "normal" package declarations; a limited-with of a renaming or package instantiation would require too much semantic analysis for the average "dumb" parser. No such limitation seemed necessary for the other two proposals, though it wasn't examined in detail. Some potentially interesting equivalences were noticed: * A "limited with" of a child has an effect that is very similar to the "type C.T;" proposal. * A type stub for a type declared in a child has an effect very similar to the "type C.T;" proposal. Because of these equivalences, it is always possible to have a single place where the access-to-incomplete type can be declared, and that place could always be the parent of the unit containing the full type, if that were desired. ************************************************************** From: Robert A. Duff Sent: Tueday, February 11, 2003 4:12 PM I'll repeat my earlier admonishment: we ought to try to get this done ASAP. The above schedule seems reasonable -- let's stick to it. I do not agree with Robert Dewar's comments last week, saying we should avoid rushing into a decision that may be technically wrong. First, we've been at this for about 8 years -- it's hardly rushing. Second, there are several workable solutions on the table. In particular, the ARG already approved one workable solution. We can patch it to address the (very minor, IMHO) concern that WG9 had, if we like. Or we can leave it as is. Or we can choose one of the other alternatives. Either way, there is little danger of passing something that's broken. It seems silly to dither about this for a long time, just because we can't decide which of the several good solutions is best. And no solution will be "perfect" in every regard. Of course, if we choose "limited with", we have to work out all the detailed rules. But that seems entirely feasible to do in a matter of months, not years. If not, we should pick one of the other good solutions. ************************************************************** From: Robert Dewar Sent: Tuesday, February 11, 2003 4:25 PM > I do not agree with Robert Dewar's comments last week, saying we should > avoid rushing into a decision that may be technically wrong. That's rather amazing, look at what you are saying, you are saying you think it's fine to rush into a decision that may be technically wrong. Do you really mean that, or, as your supporting argument implies, are you arguing that we can make a decision that is technically right? To me, the LIMITED WITH looks very promising. I had thought some consensus was gathering on that solution, and I find it disappointing that Padua thought otherwise. ************************************************************** From: Robert A. Duff Sent: Tuesday, February 11, 2003 5:00 PM > Do you really mean that, or, as your supporting argument implies, are you > arguing that we can make a decision that is technically right? I'm disagreeing with the assertion that we're "rushing" and I'm disagreeing with the assertion that there's a serious danger of making a bad choice. I claim that we *already* have a technically-good solution (type stubs). > To me, the LIMITED WITH looks very promising. I had thought some consensus > was gathering on that solution, and I find it disappointing that Padua > thought otherwise. I also have a mild preference for the "limited with" idea. But that proposal is still half baked. If we can work out the rules in a reasonable time, and we don't discover any major surprises, then I'll be happy with that. But I would also be happy with type stubs, or with Tucker's "type C.T" idea (which is pretty well baked), and if it takes two more years to "fully bake" the "limited with" idea, then I prefer the (nearly) already-worked-out solutions. If we were searching for perfection, we would still be working on the Ada 9X project. ;-) ************************************************************** From: Robert Dewar Sent: Tuesday, February 11, 2003 10:41 PM For me, the ability to reasonably automatically convert existing foreign language specs is the most important issue here. Perhaps I do not fully understand, but it seems to me that the limited with proposal is significantly better from this point of view. ************************************************************** From: Robert A. Duff Sent: Tuesday, February 11, 2003 4:50 PM Tucker wrote: > 1) When the full type declaration is visible, including > via a renaming of its enclosing package, the incomplete > type declaration is hidden from all visibility (type C.T > and limited-with proposals), or the type stub becomes equivalent to > the full type (type stub proposal). All of these proposals seem to have an issue with indirect with_clauses. Consider: package P is type C.T is tagged; procedure Proc(X: C.T); end P; package P.C is type T is tagged ...; end P.C; with P.C; package Q is Y, Z: P.C.T; end Q; with P, Q; procedure Main is begin P.Proc(X => Q.Y); -- Legal? Y := Z; -- Legal, surely. end Main; Is the above legal? Should it be? Why? The full type decl of P.C.T is not visible in Main. But we can still get our hands on that type via Y. So we've got a situation where the incomplete type and the full type T are both "there" -- only one is visible by name, but we've got to ask whether these are the same type. It seems weird (to me) to make this illegal. If it's illegal, then how do we explain why "X := Y" is legal? Tucker hates "Ripple Effects". I don't entirely agree; it seems to me that with_clauses should have been transitive in the first place. Similar question: Can Main declare a variable of type P.C.T? If we're worried about renamings, do we also need to worry about subtypes (which act like renamings of types)? E.g.: package P is type C.T is tagged; procedure Proc(X: C.T); end P; package P.C is type T is tagged ...; end P.C; with P.C; package Q is subtype S is P.C.T; end Q; with P, Q; procedure Main is W: S; -- Legal? begin Proc(X => W); -- Legal? end Main; Here, the full type decl of P.C.T is not visible, but a subtype that essentially equivalent to that full type is visible. > 2) When a package is visible, including via a renaming, any > "limited view" of the package is hidden from all visibility > (type C.T and limited-with proposals). By "limited view" we > mean the view of a package or nested package which is made > visible by the limited-with, or the child/nested package used as > a prefix in the "type C.T;" proposal. limited with P.C; package P is procedure Proc(X: C.Nest.T); end P; package P.C is package Nest is type T is tagged ...; end Nest; end P.C; with P.C; package Q is package Nest_XXX renames P.C.Nest; end Q; limited with P.C; with P, Q; procedure Main is ... end Main; Now in Main, does the above rule mean that we have a limited view of P.C, but a normal view of P.C.Nest and P.C.Nest.T? > 3) A "limited with" would not be allowed if a > full view of the package named is visible > via some other means (limited with proposal). An inherited > limited with, or an implicit limited with of a package mentioned > (but not the specifically "with"ed package itself) in a limited > with clause (e.g. "P.Q" in "limited with P.Q.R"), would be > ignored if the full view of the package is visible via some other means. Wouldn't it be simpler to use the "ignore" semantics in both cases? In fact, the previous rule about the full view hiding the limited view would cover these cases, and imply the "ignore" semantics. The above rule seems to imply that: with X; limited with X; package ... is illegal, but: limited with X; with X; package ... is legal, which seems kind of silly. > 4) No use clauses or renamings are permitted for a limited > view of a package (type C.T and limited-with proposals). > [Rationale: Use clauses seem to introduce possible Beaujolais-like > effects, and renamings of limited views just make the > "hidden from all visibility" rules even weirder.] > > 5) "Limited with" would be available only on "normal" package > declarations; a limited-with of a renaming or package instantiation > would require too much semantic analysis for the average > "dumb" parser. I don't think the issue is "dumb parser", per se. The issue is: if you say "limited with X;", the compiler has to be able to process X without looking at any source code outside of X, because that might lead back to *this* unit via a cycle. In any case, I agree with this rule, although I would state the rationale differently. >... No such limitation seemed necessary for > the other two proposals, though it wasn't examined in > detail. It seems clear that the other two proposals do not need such a limitation, because they do not require looking at the other package at all -- the type stub, or the "type C.T" tells us the name of the type, and whether it's tagged, which is all we need to know. This seems like a minor argument in favor of those other proposals -- they would allow the full type to come from a renaming or an instance. ************************************************************** From: Tucker Taft Sent: Tuesday, February 11, 2003 10:28 PM >>1) When the full type declaration is visible, including >>via a renaming of its enclosing package, the incomplete >>type declaration is hidden from all visibility (type C.T >>and limited-with proposals), or the type stub becomes equivalent to >>the full type (type stub proposal). > > > All of these proposals seem to have an issue with indirect > with_clauses. I hope not. We made a big effort to avoid the so-called "ripple" effect. This is why we talk about being visible rather than being in scope. > ... Consider: > > package P is > type C.T is tagged; > procedure Proc(X: C.T); > end P; > > package P.C is > type T is tagged ...; > end P.C; > > with P.C; > package Q is > Y, Z: P.C.T; > end Q; > > with P, Q; > procedure Main is > begin > P.Proc(X => Q.Y); -- Legal? Yes. This is thanks to the "extra" matching rules. > Y := Z; -- Legal, surely. Yes. > end Main; > > Is the above legal? Should it be? Why? The "extra" matching rules deal with the cases where the expected type is incomplete and the actual is complete, and vice-versa. > > The full type decl of P.C.T is not visible in Main. But we can still > get our hands on that type via Y. So we've got a situation where the > incomplete type and the full type T are both "there" -- only one is > visible by name, but we've got to ask whether these are the same type. > > It seems weird (to me) to make this illegal. If it's illegal, then how > do we explain why "X := Y" is legal? It is intended to be legal, thanks to the extra matching rules. > > Tucker hates "Ripple Effects". I don't entirely agree; it seems to me > that with_clauses should have been transitive in the first place. There is no ripple effect in any of these examples. And they should all be legal. > > Similar question: Can Main declare a variable of type P.C.T? No. The name "P.C.T" refers to the incomplete type in Main. > > If we're worried about renamings, do we also need to worry about > subtypes (which act like renamings of types)? No. Subtypes are treated differently by the proposed rules. There seems no implementation problem having a subtype of the complete type visible while the incomplete type declaration is also visible. Whether there is a danger of user confusion, I don't think so, but of course that is pretty hard to prove for an arbitrary user. > E.g.: > > package P is > type C.T is tagged; > procedure Proc(X: C.T); > end P; > > package P.C is > type T is tagged ...; > end P.C; > > with P.C; > package Q is > subtype S is P.C.T; > end Q; > > with P, Q; > procedure Main is > W: S; -- Legal? Yes, "S" is a subtype of a full type, and that never changes. > begin > Proc(X => W); -- Legal? Yes, the extra matching rules allow this. > end Main; > > Here, the full type decl of P.C.T is not visible, but a subtype that > essentially equivalent to that full type is visible. True. >>2) When a package is visible, including via a renaming, any >>"limited view" of the package is hidden from all visibility >>(type C.T and limited-with proposals). By "limited view" we >>mean the view of a package or nested package which is made >>visible by the limited-with, or the child/nested package used as >>a prefix in the "type C.T;" proposal. > > > limited with P.C; > package P is > procedure Proc(X: C.Nest.T); > end P; > > package P.C is > package Nest is > type T is tagged ...; > end Nest; > end P.C; > > with P.C; > package Q is > package Nest_XXX renames P.C.Nest; > end Q; > > limited with P.C; > with P, Q; > procedure Main is > ... > end Main; > > Now in Main, does the above rule mean that we have a limited view of > P.C, but a normal view of P.C.Nest and P.C.Nest.T? No. It means you have a limited view of P.C, but P.C.Nest and P.C.Nest.T are hidden from all visibility. If you want to refer to P.C.Nest, you have to do so using Q.Nest_XXX. Although it may be weird for the subpackage P.C.Nest to "disappear," this only happens in cases where the limited with was unnecessary, and a normal "with" would not have introduced circularities. We want to be sure this works, but we don't need to make it completely "pretty." >3) A "limited with" would not be allowed if a >>full view of the package named is visible >>via some other means (limited with proposal). An inherited >>limited with, or an implicit limited with of a package mentioned >>(but not the specifically "with"ed package itself) in a limited >>with clause (e.g. "P.Q" in "limited with P.Q.R"), would be >>ignored if the full view of the package is visible via some other means. > > > Wouldn't it be simpler to use the "ignore" semantics in both cases? > In fact, the previous rule about the full view hiding the limited view > would cover these cases, and imply the "ignore" semantics. Yes, disallowing the limited with is simply a "friendly" arbitrary restriction. It would be ignored in any case. But it seems friendly to notify the programmer that a "limited with" is useless. This is particularly true when the reason it is useless is because there is a visible rename. We would like the compiler to point that out. > The above rule seems to imply that: > > with X; > limited with X; > package ... > > is illegal, but: > > limited with X; > with X; > package ... > > is legal, which seems kind of silly. No, that was not the intent. The legality check on the limited with would be after fully processing the context clause (and the parent name and the inherited with's). It would not be dependent on which came first. >>... No such limitation seemed necessary for >>the other two proposals, though it wasn't examined in >>detail. > > > It seems clear that the other two proposals do not need such a > limitation, because they do not require looking at the other package at > all -- the type stub, or the "type C.T" tells us the name of the type, > and whether it's tagged, which is all we need to know. This seems like > a minor argument in favor of those other proposals -- they would allow > the full type to come from a renaming or an instance. True. ************************************************************** From: Robert A. Duff Sent: Wednesday, February 12, 2003 2:10 PM Tuck says: > I hope not. We made a big effort to avoid > the so-called "ripple" effect. This is why we > talk about being visible rather than being in scope. Could you please explain to me (again) precisely what a ripple effect is, and why you consider them to be evil? Sorry to be dense... > >>3) A "limited with" would not be allowed if a > >>full view of the package named is visible > >>via some other means (limited with proposal). An inherited > >>limited with, or an implicit limited with of a package mentioned > >>(but not the specifically "with"ed package itself) in a limited > >>with clause (e.g. "P.Q" in "limited with P.Q.R"), would be > >>ignored if the full view of the package is visible via some other means. > > > > Wouldn't it be simpler to use the "ignore" semantics in both cases? > > In fact, the previous rule about the full view hiding the limited view > > would cover these cases, and imply the "ignore" semantics. > > Yes, disallowing the limited with is simply a "friendly" > arbitrary restriction. It would be ignored in any case. > But it seems friendly to notify the programmer that a > "limited with" is useless. This is particularly true when > the reason it is useless is because there is a visible rename. > We would like the compiler to point that out. This is a fairly minor point, but I still suggest going with the simpler rule. The RM is not in the business of good error messages; leave that to implementers. It's like the use-clause thing -- if you have two homographs called X they cancel each other out. The straightforward implementation of that would cause the compiler to say "no directly visible X" or something, which is an extremely unhelpful message. Good compilers go to some extra trouble to point out that there are actually *two* X's, each of which is "almost" directly visible. Maybe when you see the actual RM wording, you'll agree... ;-) ************************************************************** From: Tucker Taft Sent: Wednesday, February 12, 2003 3:18 PM Robert A Duff wrote: > > Tuck says: > > > I hope not. We made a big effort to avoid > > the so-called "ripple" effect. This is why we > > talk about being visible rather than being in scope. > > Could you please explain to me (again) precisely what a ripple effect > is, and why you consider them to be evil? Sorry to be dense... A ripple effect is when adding or removing a "with" clause in some unit on which the current unit depends semantically (potentially quite indirectly) has a significant effect on the legality of the current unit. For example, at one point in Ada 83, to use 'Address, the current unit had to have a semantic dependence on System. This involves a ripple effect, since adding or removing a "with" of System in some "distant" unit could affect legality of the unit containing the 'Address. > > >>3) A "limited with" would not be allowed if a > > >>full view of the package named is visible > > >>via some other means (limited with proposal). An inherited > > >>limited with, or an implicit limited with of a package mentioned > > >>(but not the specifically "with"ed package itself) in a limited > > >>with clause (e.g. "P.Q" in "limited with P.Q.R"), would be > > >>ignored if the full view of the package is visible via some other means. > > > > > > > > > Wouldn't it be simpler to use the "ignore" semantics in both cases? > > > In fact, the previous rule about the full view hiding the limited view > > > would cover these cases, and imply the "ignore" semantics. > > > > Yes, disallowing the limited with is simply a "friendly" > > arbitrary restriction. It would be ignored in any case. > > But it seems friendly to notify the programmer that a > > "limited with" is useless. This is particularly true when > > the reason it is useless is because there is a visible rename. > > We would like the compiler to point that out. > > This is a fairly minor point, but I still suggest going with the simpler > rule. The RM is not in the business of good error messages; leave that > to implementers. It seems very simple to disallow these useless "limited withs," so I would prefer to make them illegal, and my sense was that the rest of the ARG in Padua felt similarly. But they can presumably answer for themselves. > ... > Maybe when you see the actual RM wording, you'll agree... ;-) With whom? ************************************************************** From: Tucker Taft Sent: Tuesday, February 11, 2003 2:21 PM Here is some initial analysis of implementation issues associated with "limited with." I tried to make most of the analysis technology independent, but of course that is nearly impossible, so it may apply less or more to technologies other than AdaMagic. -Tuck ----------------- Thoughts on the implementation of "limited with" in AdaMagic $Revision: 1.9 $ $Date: 2003/09/30 02:01:17 $ The "limited with" clause makes a "limited view" of a package visible in the compilation unit that has the limited-with clause, as well as all of its descendants. However, the limited view is hidden from all visibility (as are all the incomplete types and nested package limited views within it) under certain circumstances. After much discussion at the Padua ARG meeting, the best rule seemed to be that if the full view of the package is visible, including via one or more renamings, then the limited view is hidden from all visibility. [This was considerd necessary so as to avoid having two different views of the same package usable at the same point in the source text, which was felt to create both implementation complexity and possible user confusion.] SEPARATE "LIMITED" COMPILATION UNIT Our model for implementing a limited view of a library package will be that it has its "own" compilation-unit name, constructed from its "real" name but with suffix "'Limited". Hence P.Q.R'Limited is the limited view of P.Q.R. [Note that "P.Q.R.Limited" could also be used as the name, since "limited" is a reserved word, and would never be the real name of a package. However, it is very important that in any case P.Q.R.Limited or P.Q.R'Limited not depend semantically on P.Q.R itself.] In the representation of the library, it is important that P.Q.R'Limited can exist before P.Q.R has been fully compiled, and can retain some continuing existence while P.Q.R is being fully compiled, and thereafter. It is also important to realize that the packages nested within P.Q.R'Limited are themselves limited views. Hence, P.Q.R'Limited.NP always denotes a limited view of a package, if P.Q.R.NP denotes a nested package spec. Similarly, P.Q.R'Limited.T denotes an incomplete type declaration for T, if P.Q.R.T denotes a full type declaration. SEPARATE TIMESTAMP FOR LIMITED VIEW P.Q.R'Limited has its own timestamp, for the purposes of out-of-dateness checking. It is important that this *not* change when a "full" compile of P.Q.R is started, because that would make the units that P.Q.R depends on that themselves depend on P.Q.R'Limited, automatically appear out of date. Once the full compile of P.Q.R is complete, a check for consistency between P.Q.R'Limited and P.Q.R should be performed. If the views are no longer consistent, due to some change in P.Q.R, a new version of P.Q.R'Limited should be constructed, and its timestamp changed. This will effectively put all the dependents of P.Q.R'Limited out of date, and may in fact put P.Q.R itself out of date. If so, then P.Q.R will have to be compiled again. Ideally, this will be performed immediately, since the source code for P.Q.R is readily available at that time. However, depending on the compilation mechanism, it may be first necessary to compile other dependents of P.Q.R'Limited, so they are back in an up-to-date state before re-attempting the full compilation of P.Q.R. This might be left until link time, or to a separate "make" tool. Of course if P.Q.R'Limited is initially created by simply parsing the source text for P.Q.R, the timestamp of P.Q.R'Limited would be set at parse time. For those compilation systems that retain the date of modification of the source file as the time stamp of the corresponding units in the program library, some of this becomes simpler. Nevertheless, it seems a helpful, and in some cases necessary, "optimization" to *not* change the timestamp of the limited view every time the source text timestamp changes. Because the check for consistency between a full view and a limited view should be relatively easy to perform, there seems no need to change the timestamp on the limited view so long as it remains consistent with the full view, even if the full view corresponds to source text with a different timestamp. This means that units that depend only on the P.Q.R'Limited are isolated from "minor" changes to P.Q.R, and are only affected when a type or nested package is added or removed from P.Q.R. In other words, you get a "poor man's" version of incremental compilation as a side effect of implementing limited views of packages. HIDING THE LIMITED VIEW As mentioned above, a limited view of a package can be hidden from all visibility. The anticipated rule is that the limited view is hidden from all visibility if the full view is visible either "normally" or via a rename. This implies that if P.Q.R is visible, including via a rename, then P.Q.R'Limited is not visible. Furthermore, even when P.Q.R'Limited is visible, it is possible that P.Q.R'Limited.NP might not be visible, because P.Q.R.NP might be visible via a renaming. A relatively simple implementation of this requirement would seem to be to build up a set representing those (external) packages that have a full view visible, including via a rename, within the current compilation unit. (By "external" we mean it is from some other compilation unit.) This set will not grow once the context clause has been fully processed, since it is not possible to locally declare a renaming of a package that is not already visible. This set of visible packages would probably be represented using a hash table of some sort. It would start out empty when starting to compile a new compilation unit, and would grow each time a (real, not limited) "with" for a package or a package renaming is processed. Each of these "with"ed packages would be scanned for enclosed renamings, recursively. [Alternatively, the representation of every package could include a set of all enclosed renamings (directly or indirectly).] The "with"ed package, plus all of the packages for which it contains a renaming, would be unioned into this set. Later, whenever a limited view of a package is encountered within a name, a check is made to see whether a corresponding full view exists in this set. If so, the limited view may not be named, and so the name is illegal. Another anticipated rule is that a limited with clause is illegal if the package is already visible, including via a rename. This would seem to be friendlier than allowing the programmer to put a "limited with" on a compilation unit and then not be able to use it at all. However, there will still be "limited with"s inherited from ancestor units, and these will be essentially ignored if the corresponding package is fully visible some other way. Also, in a limited with clause like "limited with P.Q.R;" presumably implicit limited withs for P and P.Q are provided, but such an implicit limtied with again must be ignored if the package mentioned is already fully visible via some other means. To enforce the above rule regarding disallowing and/or ignoring limited withs, it may be necessary to do a second pass over the context clause, after the full set of visible packages has been built up. In many cases, a second pass is already necessary in Ada 95 to correctly deal with "with"s of private units, since it is not known whether those with's are legal until the name of the compilation unit being compiled is known, so it may be possible to combine these error-checking passes. WHEN IS THE INCOMPLETE TYPE COMPLETE? Our current plan is to disallow "use" clauses and renamings for limited views. Given that, the only way to name one of the incomplete types inside a limited view is by naming the enclosing package, so there is no need to have a separate mechanism for determining whether the incomplete type declaration is visible. It is visible exactly when the enclosing limited view can be named. However, there are access types that might have been declared using the incomplete type, and it is important to know when a dereference of the access type is considered "complete." The basic rule would be that when the full type is visible, including via a rename of the enclosing package, (or equivalently, when the incomplete type declaration is hidden from all visibility), the access type may be dereferenced to produce the full type. This implies that any time such an access type is dereferenced, a check should be made whether the designated type's enclosing limited view(s) are all still visible. If they are all still visible, then the incomplete type is still visible, and the dereference produces an incomplete object. If any one of them is hidden from all visibility, then the full type is visible, and the dereference produces an object of a "full" type. Some optimization of this check is probably possible. Even when a dereference of the access type is considered incomplete, it may still be legal in a context where a full type would normally be required, so long as the expected type is the full type, or the full type is "nearby." One of the versions of the type-stub proposal laid out the rules for this matching between incomplete and full type, and those rules seem still relevant for all of the other proposals as well. Here is a copy of those rules (from AI-00217-04/04): A dereference (implicit or explicit) of a value of an access type whose designated type D is incomplete is allowed only in the following contexts: * in a place where the completion of D is available (see above); * in a context where the expected type is E and o E covers the completion of D, o E is tagged and covers D, o E covers D'Class or its completion, or o E'Class covers D or its completion; * as the target of an assignment_statement where the type of the value being assigned is V, and V or V'Class is the completion of D. In these contexts, the incomplete type is defined to be the same type as its completion, and its first subtype statically matches the first subtype of its completion. The first bullet would be replaced by "where the full type declaration for D is visible" for the limited-with proposal (and for the "type C.T;" proposal). ************************************************************** From: Pascal Leroy Sent: Thursday, March 6, 2003 4:56 AM (I am discussing the issue below in the context of limited with, but I believe that the other incarnations of 217 all have similar problems if they use visibility to determine when the completion can be used.) A while back, in a message about implementation of limited withs in AdaMagic, Tuck wrote: > As mentioned above, a limited view of a package can be hidden from > all visibility. The anticipated rule is that the limited > view is hidden from all visibility if the full view is visible > either "normally" or via a rename. This implies that if > P.Q.R is visible, including via a rename, then P.Q.R'Limited > is not visible. Furthermore, even when P.Q.R'Limited > is visible, it is possible that P.Q.R'Limited.NP might > not be visible, because P.Q.R.NP might be visible via > a renaming. From a user's perspective, the notion that you have a limited view of P.Q.R but a full view of its nested package P.Q.R.NP sounds very confusing to me. Especially considering that the reason why you have a full view of NP is that someone somewhere declares a renaming of NP. From an implementer's perspective, my head aches when I try to see how we could implement this. In fact, the above description seems distinctly at odds with what was discussed during the meeting. The minutes say: "Randy says that two views of the same package make the implementation much harder. Our implementation only expects a single view with a few well-defined places of visibility change. This is scattered all about. Pascal expresses agreement with Randy's assessment." And then the minutes present rules intended to avoid the "two views" issue (although arguably the problem discussed by Tucker was not addressed, at least as far as I remember). The implementation difficulties for us are worse than I originally thought because of incremental compilation. Say that you compile a unit containing the name P.Q.R.NP.Var, and that name is legal because of the existence of some renaming of NP. Then that renaming is removed. During incremental recompilation the name P.Q.R.NP.Var must be obsolesced, even though it doesn't reference the renaming that is being removed. That's hard. Not as hard as a full-fledged ripple effect maybe, but still hard. I understand that there is some similarity with name clashes introduced by use clauses, but I'd rather not add more complexity to code that is already very hairy. Also note that from a user's perspective name clashes are probably quite rare, but visibility changes due to the introduction/removal of renamings might be much more frequent, and would certainly result in much puzzlement. This leads me to two questions: 1 - Why did we decide to go for visibility anyway? What would be wrong with using the "availability" defined in 217-05? 2 - If we have to stick to visibility, couldn't we define the rules in such a way that there is only one view visible? I'd say that the situation described by Tucker above should be illegal, and that if (1) you have a limited with of P.Q.R and (2) some renaming gives you a full view of P.Q.R.NP then your code is illegal. You need to add a "with P.Q.R" to hide the limited view of P.Q.R from all visibility. Comments? ************************************************************** From: Tucker Taft Sent: Thursday, March 6, 2003 8:17 AM Pascal Leroy wrote: ... > From a user's perspective, the notion that you have a limited view of P.Q.R but > a full view of its nested package P.Q.R.NP sounds very confusing to me. > Especially considering that the reason why you have a full view of NP is that > someone somewhere declares a renaming of NP. It's not quite that bad. Remember that you don't have a full view of NP using the name "P.Q.R.NP". You have to use the name introduced via the renaming. What you *lose* is the ability to refer to the limited view of NP via the name "P.Q.R.NP." ... > The implementation difficulties for us are worse than I originally thought > because of incremental compilation. Say that you compile a unit containing the > name P.Q.R.NP.Var, and that name is legal because of the existence of some > renaming of NP. This is not what I intended to propose. Var would never be visible via the name "P.Q.R.NP.Var" if you have a limited view of P.Q.R, since objects are not included in the limited view. The presence of a renaming only *removes* something from the limited view, it never adds something. > ... Then that renaming is removed. During incremental recompilation > the name P.Q.R.NP.Var must be obsolesced, even though it doesn't reference the > renaming that is being removed. That's hard. Not as hard as a full-fledged > ripple effect maybe, but still hard. Can you consider this again with hopefully a clearer explanation of what was intended? ... > This leads me to two questions: > > 1 - Why did we decide to go for visibility anyway? What would be wrong with > using the "availability" defined in 217-05? I think if you study the availability rules, you will find they are essentially the same as the proposed visibility rules, but just couched in different terms. If you find a significant difference, I would like to know what it is. > 2 - If we have to stick to visibility, couldn't we define the rules in such a > way that there is only one view visible? I'd say that the situation described > by Tucker above should be illegal, and that if (1) you have a limited with of > P.Q.R and (2) some renaming gives you a full view of P.Q.R.NP then your code is > illegal. You need to add a "with P.Q.R" to hide the limited view of P.Q.R from > all visibility. I'm not sure what you mean here, since we started off with different interpretations of the proposed rule. Could you restate this, and contrast it with what I have explained above to be the intent, namely that a rename causes things to be removed from a limited view, rather than being added. ************************************************************** From: Pascal Leroy Sent: Friday, March 7, 2003 10:27 AM > This is not what I intended to propose. Var would never be visible > via the name "P.Q.R.NP.Var" if you have a limited view of > P.Q.R, since objects are not included in the limited view. > The presence of a renaming only *removes* something from > the limited view, it never adds something. Ok, I was confused, but I still have essentially the same problems. For clarity, let's look at some piece of code: package P.Q.R is package NP is type T is new Boolean; end NP; end P.Q.R; with P.Q.R; package A is package Ren renames P.Q.R.NP; end A; limited with P.Q.R; -- with A; procedure Main is type Acc is access P.Q.R.NP.T; begin null; end Main; If I understand you correctly, this code is legal. However, if the "with A" in Main is uncommented, the declaration of type Acc becomes illegal, because P.Q.R.NP is hidden from all visibility. From a user's perspective, I still find this rule mysterious, but maybe it's not going to happen frequently in practice. From an implementer's perspective I still have to have two views around: P.Q.R'Limited for all of P.Q.R except P.Q.R.NP, and P.Q.R'Full for P.Q.R.NP (but only when it is accessed through the renaming). It means that there are a zillion places in my compiler where I need to ask the question: which view do I see? This is feasible, but I'm sure I won't get it right the first time (or the second time). Now looking at incremental compilation again, you have not solved my problem. Say that we had the "with A" all along, but initially A didn't include the renaming. Then someone adds that bloody renaming. The name P.Q.R.NP.T in Main must be obsolesced, even though it doesn't reference A. Let me say it once more: that's hard. > I think if you study the availability rules, you will find > they are essentially the same as the proposed visibility > rules, but just couched in different terms. If you find > a significant difference, I would like to know what it is. It's hard to tell for sure because the availability rules in 217-05 are formulated in terms of type stubs. But if you try to adapt them for "limited withs", it would seem that you'd have to say that the completion of T is available (1) within the extended scope of the completion of T and (2) within the scope of a (nonlimited) with clause for the package that declares T. I fail to see how this would cause a "with A" to hide P.Q.R.NP from all visibility. In fact I fail to see how the presence of "with A" would have any bearing on the visibility of entities exported by P.Q.R. It seems to me that if we used availability, the completion of T would not be available in Main (regardless of whether there is a "with A" or not) and therefore we would have only one view of P.Q.R, the limited one. > I'm not sure what you mean here, since we started off with > different interpretations of the proposed rule. Could you > restate this... Forget it, the scheme that I had in mind would introduce ripple effects. ************************************************************** From: Tucker Taft Sent: Friday, March 7, 2003 12:12 PM > If I understand you correctly, this code is legal. However, if the "with A" > in Main is uncommented, the declaration of type Acc becomes illegal, because > P.Q.R.NP is hidden from all visibility. Right. > From a user's perspective, I still find this rule mysterious, but maybe it's > not going to happen frequently in practice. It shouldn't, because it only happens if a piece of code has a limited-with on something on which it depends semantically. ... > It's hard to tell for sure because the availability rules in 217-05 are > formulated in terms of type stubs. But if you try to adapt them for "limited > withs", it would seem that you'd have to say that the completion of T is > available (1) within the extended scope of the completion of T Unless we change the definition of scope, this would include all semantic dependents of the package where the completion is declared (see 8.2(10)). So this would create ripple effects and would be even worse, as far as incremental recompilation, I presume. I really think you need to address 8.3(19) which says within the *scope* of the completion, the first declaration is hidden from all visibility. I have proposed changing this to say where the completion is visible (including via a renaming), the first declaration is hidden from all visibility. We can fiddle with these words further, since they really only affect this new case. > ... and (2) within > the scope of a (nonlimited) with clause for the package that declares T. I fail > to see how this would cause a "with A" to hide P.Q.R.NP from all visibility. In > fact I fail to see how the presence of "with A" would have any bearing on the > visibility of entities exported by P.Q.R. > > It seems to me that if we used availability, the completion of T would not be > available in Main (regardless of whether there is a "with A" or not) and > therefore we would have only one view of P.Q.R, the limited one. I think you are perhaps muddying things by talking about "availability" since that is a term that was introduced in an AI and never fully explored. Let's focus on this renaming thing. I was trying to address the concern that you didn't want to have two views of the same package at the same place. I presumed that referred to sub-packages as well. If a renaming does not remove NP from the limited view of P.Q.R, then you can see the limited view of P.Q.R.NP through that name, and the full view of the same package via the renaming. I can live with that, but I thought we were trying to avoid that. Let's decide that question, and then construct rules that accomplish what we want. We can call them "visibility" or "availability" or "supercalifragility" rules or whatever once we decide what the rules should be. ************************************************************** From: Pascal Leroy Sent: Friday, March 7, 2003 2:58 PM > Let's focus on this renaming thing. I was trying to address > the concern that you didn't want to have two views of the same package > at the same place. I presumed that referred to sub-packages as well. > If a renaming does not remove NP from the limited view of P.Q.R, then > you can see the limited view of P.Q.R.NP through that name, > and the full view of the same package via the renaming. I can live > with that, but I thought we were trying to avoid that. I actually had a much more stringent requirement in mind: ideally, in the course of compiling a compilation unit, there would be only one view of each and every (compilation) unit in its closure. If we manage to achieve this, it would simplify the implementation considerably. So for instance when compiling Main in my example, you would either need to open P.Q.R'Full or P.Q.R'Limited, but not both. And you would probably know that early in the compilation process, e.g. after processing the context clauses. Regarding the incremental compilation issue, the legality of a name should not be affected by adding/removing random declarations to random units (in particular, package renamings). Unsure how/if we can come up with rules that satisfy these requirements. >We can call them "visibility" or "availability" or "supercalifragility"... :-) :-) ************************************************************** From: Randy Brukardt Sent: Saturday, March 22, 2003 12:36 AM This example shows two ways that I would use alternative #6 of AI-217 in the Claw Builder. See the similar write-up on alternative #5 for details about the Builder and why it is important to be able to use this facility in it. [Editor's note: Here are those details.] The Claw Builder is a real, existing program which could use this facility. The problem is that some objects need references to other types of objects, and these needs are circular. For instance, (some) types of window objects include menu objects. And some types of menu objects include actions that open a window. The current Claw Builder solves this problem by using names rather than access objects to connect the objects in some cases. This is usually done only where necessary to break circularities. For instance, menu objects name the windows they are to open, rather than linking to them. Using names causes several problems: -- Accesses to the linked object is much slower, as they have to be looked up by name before use; -- If the user renames the linked object, we have to walk the entire project to insure any names are updated; -- If the user copies the linked object and then renames the copy (the required behavior), we have to be careful NOT to walk the project and update names -- harming code reuse. -- We can't have overloaded names (not a problem for windows, but can happen in other cases). A root window object is an abstract object with a fairly large set of operations. Each concrete object has to provide implementations for many of these operations (some it can inherit). All of these operations are dispatching. Typically, a user of the operation would apply it to a list of windows using an iterator generic, with the operation dispatching to the correct implementation. For the purposes of this discussion, we'll look at just a few: Show, Hide, Display_Name. The existing package looks something like: with CBuild_Menu; package CBuild_Root is type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with private; type Any_Window_Access_Type is access all Root_Window_Type'Class; procedure Show (Window : in out Root_Window_Type) is abstract; procedure Hide (Window : in out Root_Window_Type) is abstract; function Display_Name (Window : in Root_Window_Type) return String is abstract; ... private type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with record ... CBuild_Menu.Something ... end record; end CBuild_Root; The menu package looks something like (greatly simplified): with CBuild_Id_Type; package CBuild_Menu is type Menu_Item is record Name : String (1..20); Action : Action_Type; Dialog : CBuild_Id_Type; -- The name of a dialog window. -- If Action=Open_Dialog. end record; procedure Simulate_Action (Item : in Menu_Item); end CBuild_Menu; with CBuild_Root, CBuild_Lists; package body CBuild_Menu is procedure Simulate_Action (Item : in Menu_Item) is begin if Item.Action = No_Action then null; elsif Item.Action = Open_Dialog then CBuild_Root.Show ( CBuild_Lists.Lookup(Item.Dialog, CBuild_Data.Top_Level_Window_List)); ... -- Other actions. end if; end Simulate_Action; end CBuild_Menu; [End details copied from alternative #5.] In order to directly use a Any_Window_Access_Type instead of the name in CBuild_Menu, we would need restructure the CBuild_Root package. We'd need to construct an abstract of this package for use in circular definitions. This is necessary to have a single access type, which avoids the need to use a conversion on nearly every use. Using alternative #7 (and assuming that AI-326 is approved), this could look like: limited with CBuild_Root; package CBuild_Root_Abstract is type Any_Window_Access_Type is access all CBuild_Root.Root_Window_Type'Class; end CBuild_Root_Abstract; This abstract would be used only when you need to declare instances (usually components) of the type in locations where the reference would be circular. Since we don't want an extra access type, the real package would be modified to: with CBuild_Root_Abstract; with CBuild_Menu; package CBuild_Root is type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with private; subtype Any_Window_Access_Type is CBuild_Root_Abstract.Any_Window_Access_Type; procedure Show (Window : in out Root_Window_Type) is abstract; procedure Hide (Window : in out Root_Window_Type) is abstract; function Display_Name (Window : in Root_Window_Type) return String is abstract; ... private type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with record ... CBuild_Menu.Something ... end record; end CBuild_Root; The client package (in this case, for menus) would look something like: with CBuild_Root_Abstract; package CBuild_Menu is type Menu_Item is record Name : String (1..20); Action : Action_Type; Dialog : CBuild_Root_Abstract.Any_Window_Access_Type; -- If Action=Open_Dialog. end record; procedure Simulate_Action (Item : in Menu_Item); end CBuild_Menu; with CBuild_Root; package body CBuild_Menu is procedure Simulate_Action (Item : in Menu_Item) is begin if Item.Action = No_Action then null; elsif Item.Action = Open_Dialog then CBuild_Root.Show (Item.Dialog.all); ... -- Other actions. end if; end Simulate_Action; end CBuild_Menu; As I've said twice before, I don't much like this solution, because it's not clear when to use the abstract package, and when to use the main package. It would be preferable to keep an understandable separation between them. One way to do this is create a "client" package and a "creator" package. The "client" package would be used by ordinary client packages of the abstraction. The "creator" package would be used only by packages that need to create new extensions of the type or create objects of the type. The "creator" package would, of course, be similar to the existing package. The "client" package would provide enough operations that regular clients could use it only. Since the "client" package would be used most frequently, I've given it the existing name. limited with CBuild_Root_Definition; package CBuild_Root is type Any_Window_Access_Type is access all CBuild_Root_Definition.Root_Window_Type'Class; procedure Show (Window : in out CBuild_Root_Definition.Root_Window_Type'Class); procedure Hide (Window : in out CBuild_Root_Definition.Root_Window_Type'Class); function Display_Name (Window : in CBuild_Root_Definition.Root_Window_Type'Class) return String; ... end CBuild_Root; One annoyance here is the need to give the fully expanded name of the incomplete type, over and over. That's because use clauses are prohibited on the limited withed package. But I'll live. :-) The "creator" package would be as in the previous example (except for the name): with CBuild_Root; with CBuild_Menu; package CBuild_Root_Definition is type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with private; subtype Any_Window_Access_Type is CBuild_Root.Any_Window_Access_Type; procedure Show (Window : in out Root_Window_Type) is abstract; procedure Hide (Window : in out Root_Window_Type) is abstract; function Display_Name (Window : in Root_Window_Type) return String is abstract; ... private type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with record ... CBuild_Menu.Something ... end record; end CBuild_Root_Definition; The body of CBuild_Root would need visibility on the completing type so that it could implement the procedures: with CBuild_Root_Definition; package body CBuild_Root is procedure Show (Window : in out CBuild_Root_Definition.Root_Window_Type'Class) is begin CBuild_Root_Definition.Show (Window); end Show; procedure Hide (Window : in out CBuild_Root_Definition.Root_Window_Type'Class) is begin CBuild_Root_Definition.Hide (Window); end Hide; function Display_Name (Window : in CBuild_Root_Definition.Root_Window_Type'Class) return String is begin return CBuild_Root_Definition.Display_Name (Window); end Display_Name; ... end CBuild_Root; With this structure, the clients with CBuild_Root, and the concrete object packages with (or are children of) CBuild_Root_Definition. The two packages have clearly defined roles. With these packages, the use of the types in the menu package is the same for all three alternatives: with CBuild_Root; package CBuild_Menu is type Menu_Item is record Name : String (1..20); Action : Action_Type; Dialog : CBuild_Root.Any_Window_Access_Type; -- If Action=Open_Dialog. end record; procedure Simulate_Action (Item : in Menu_Item); end CBuild_Menu; package body CBuild_Menu is procedure Simulate_Action (Item : in Menu_Item) is begin if Item.Action = No_Action then null; elsif Item.Action = Open_Dialog then CBuild_Root.Show (Item.Dialog.all); ... -- Other actions. end if; end Simulate_Action; end CBuild_Menu; --- Of course, the best way to avoid the issue of deciding when to use one of the two packages is to avoid having the second one in the first place. The primary reason we need this package is because of access type proliferation problems. If the solution in AI-230 is adopted, we don't have that problem. In that case, we can put the stub directly in the menus package. In that case, the CBuild_Root package need not be modified at all: with CBuild_Menu; package CBuild_Root is type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with private; type Any_Window_Access_Type is access all Root_Window_Type'Class; procedure Show (Window : in out Root_Window_Type) is abstract; procedure Hide (Window : in out Root_Window_Type) is abstract; function Display_Name (Window : in Root_Window_Type) return String is abstract; ... private type Root_Window_Type is abstract new Ada.Finalization.Limited_Controlled with record ... CBuild_Menu.Something ... end record; end CBuild_Root; Then, the menu package would declare the stub and an anonymous access type: limited with CBuild_Root; package CBuild_Menu is type Menu_Item is record Name : String (1..20); Action : Action_Type; Dialog : access CBuild_Root.Root_Window_Type'Class; -- If Action=Open_Dialog. end record; procedure Simulate_Action (Item : in Menu_Item); end CBuild_Menu; with CBuild_Root; package body CBuild_Menu is procedure Simulate_Action (Item : in Menu_Item) is begin if Item.Action = No_Action then null; elsif Item.Action = Open_Dialog then CBuild_Root.Show (Item.Dialog.all); ... -- Other actions. end if; end Simulate_Action; end CBuild_Menu; This solution clearly perturbs the existing program the least, so it is preferred if it is available. It's interesting to note that alternatives #5 and #6 look essentially the same on this program. The structure of the program is the same, presuming the same assumptions about the language features employed. The advantage of alternative #5 is to be able to give a different, visible name to the stub (which simplifies the rules and probably the implementation, but which I didn't take advantage of in this moderately sized example). The advantage of #6 is not to have to declare the stub at all. Alternative #7 perturbs the structure of the program further, and can't really handle the last example at all. That makes it a distant third in my view. Alternative #6 probably has the edge on ease of use, but I'd like to see a worked-out version whose visibility/availability rules really work before I could even give it a partial endorsement. (And it always will be much more expensive to implement.) That concludes my look at implementing the Claw Builder using these alternatives. One more piece of homework done. Yeah!!! ************************************************************** From: Pascal Leroy Sent: Friday, March 21, 2003 8:31 AM > That means, in fact, that visibility IS the right model here, as long as > it is clear that it is the visibility of the original completing > declaration that we are talking about, and not the visibility of some subtype > name that happens to be declared somewhere else. The more I read chapter 8 (and the mail messages that Tuck and you wrote) the more confused I am. Consider the example: package P is type T is new Boolean; end P; with P; package Q is end Q; limited with P; with Q; package R is X : P.T; -- Legal? end R; The question is: is the limited (or incomplete if you prefer) view of P.T hidden from all visibility in R? If I read Tuck's proposed change to 8.3(19) it seems that the answer is a resounding Yes. The reason is that 8.3(14) says that "a declaration is visible within its scope, except when hidden from all visibility." R is clearly part of the scope of P.T, because Q is a semantic dependent of P and R is a semantic dependent of Q. And none of the "hidden from all visibility" rules apply to P.T here (see 8.3(15-20)). So the completion of P.T is visible in R, and the incomplete view of P.T is hidden from all visibility. None of this is new, but in Ada 95 (where you don't have "limited with") P is hidden from all visibility within R because of 8.3(20), so even though P.T is visible in R, it cannot be named (wonderful language!). However the "limited with" clause clearly makes (the limited view of) P visible in R, and so P.T can be named, and the declaration of X is legal. But of course there is an awful ripple effect here, because removing the "with P" in Q causes R to become illegal. Yuck. In essence, I guess I am saying that visibility has the same transitivity effect as scope as far as I can tell. ************************************************************** From: Tucker Taft Sent: Friday, March 21, 2003 9:28 AM > The question is: is the limited (or incomplete if you prefer) view of > P.T hidden from all visibility in R? If I read Tuck's proposed change > to 8.3(19) it seems that the answer is a resounding Yes. That was certainly not my intent. I have suggested that 8.3(19) be changed so that the hidden-from-all-visibility only happens where the completion is *visible* as opposed to throughout the scope of the completion. In the above, the completion is *not* visible inside R, so the incomplete declaration is not hidden. But as I read on... > The reason is that 8.3(14) says that "a declaration is visible within > its scope, except when hidden from all visibility." R is clearly part > of the scope of P.T, because Q is a semantic dependent of P and R is a > semantic dependent of Q. And none of the "hidden from all visibility" > rules apply to P.T here (see 8.3(15-20)). So the completion of P.T is > visible in R, and the incomplete view of P.T is hidden from all > visibility. Hmmmm.... I see your point. I was presuming that if there is no way to name the enclosing package, then all the declarations in the visible part of the package are similarly hidden from all visibility. The RM doesn't back up that view. It looks like in addition to changing 8.3(19) we would have to also change 8.3(14) roughly as follows: A declaration is visible within its immediate scope except where hidden from all visibility. A declaration that occurs immediately within the visible part of an enclosing visible declaration, or that is a public child of a visible declaration, is visible except where hidden from all visibility. A declaration is hidden from all visibility as follows: > > None of this is new, but in Ada 95 (where you don't have "limited with") > P is hidden from all visibility within R because of 8.3(20), so even > though P.T is visible in R, it cannot be named (wonderful language!). > However the "limited with" clause clearly makes (the limited view of) P > visible in R, and so P.T can be named, and the declaration of X is > legal. > > But of course there is an awful ripple effect here, because removing the > "with P" in Q causes R to become illegal. Yuck. > > In essence, I guess I am saying that visibility has the same > transitivity effect as scope as far as I can tell. Good point. As suggested above, 8.3(14) would also have to be changed to make the change to 8.3(19) useful. ************************************************************** From: Pascal Leroy Sent: Friday, March 21, 2003 3:28 PM > Hmmmm.... I see your point. I was presuming that if there is > no way to name the enclosing package, then all the declarations > in the visible part of the package are similarly hidden from > all visibility. The RM doesn't back up that view. It looks > like in addition to changing 8.3(19) we would have to also > change 8.3(14) roughly as follows: > > A declaration is visible within its immediate scope except > where hidden from all visibility. A declaration that occurs immediately > within the visible part of an enclosing visible declaration, or that > is a public child of a visible declaration, is visible except where > hidden from all visibility. A declaration is hidden from > all visibility as follows: The second sentence doesn't seem to solve the problem. It says in essence "A declaration ... is visible except where hidden from all visibility". That's not saying where it's visible (I presume that you don't want to say that it's visible to the entire universe, even outside its scope). I would have expected a qualification like "is visible except where hidden from all visibility." I'm not sure what to put in the <>, though. Maybe "is visible at the places where the enclosing declaration or parent unit is visible, except ..."? ************************************************************** From: Tucker Taft Sent: Friday, March 21, 2003 4:02 PM We can just fall back on "scope" I think. I.e.: Outside its immediate scope, but within its (extended) scope, a declaration that is not hidden from all visibility is visible if the declaration of the immediately enclosing declarative region, or a renaming thereof, is visible. But this may be overkill, if we adopt one of the more restrictive versions of 8.3(19) that I have suggested. E.g., if we say that: a declaration is hidden from all visibility within the immediate scope of its completion, and within the extended scope of the completion where the declaration of the (completion's) enclosing library unit, or a library unit renaming thereof, is visible. Or something like that... ************************************************************** From: Randy Brukardt Sent: Monday, March 24, 2003 3:31 PM BTW, Pascal, how is AI-217-06 coming? I've just been thinking that the 'two views' problem is impossible to avoid without serious ripple effects. If that's true, of course, 'limited with' is dead in the water, because it will be even harder to implement than previously thought (I'd say impossible, but of course it's always possible to implement anything self-consistent.) The problem is that you have a 'view' of a package in your symboltable any time the package is in the semantic closure. Thus, any rule that attempts to avoid 'two views' issue is going to cause ripple effects. For example: package P is type Root is abstract tagged ...; end P; with P; package Q is type Win is new Root with ... end Q; with Q; package R is Main : Q.Win; end R; limited with P; with Q, R; package S is type Any_Root is access all P.Root; -- Legal? Child : Q.Win := R.Main; end S; Clearly, we have a full view of P in our semantic closure, so all of the information about type Root will be loaded (but not visible). That's clearly necessary so we can handle the initialization of Child. (In Janus/Ada, the entire symboltable for the package is loaded in this case, but I suppose one could arrange to load only a partial view - but that view necessarily has quite a bit more information that the 'limited view' does.) The rules that we've discussed for fixing this problem is that the limited with would be either illegal or ignored in some case. In order to avoid having two views, the rule clearly has to be effective any time the package P is in the semantic closure (as in this example). In that case, either the 'limited with P' is illegal (but that seems confusing to the user, as there is no sign of any P in package S), or the 'limited with P' is ignored, meaning the 'P.Root' is illegal, since there is no P visible (which is even more confusing to the user). Moreover, if maintenance changes Q to not depend on P, then of course S becomes legal: a classic ripple effect. Of course, the ripple effect is bad if it occurs in the other direction. Presume that a fielded system was a package Q that doesn't depend on P. S is legal. Then, if maintenance adds a 'with P' to package Q, S becomes illegal. (And S would be illegal even if Child wasn't declared and package Q wasn't withed by S.) Ripple effects like this can't be tolerated. Thus, it appears to me that we cannot legislate away the 'two views' problem. That makes the implementation cost of this proposal far more than we thought in Padua (and that was higher than the other proposals to begin with). ************************************************************** From: Pascal Leroy Sent: Wednesday, March 26, 2003 8:49 AM > I've just been thinking that the 'two views' problem is impossible to > avoid without serious ripple effects. If that's true, of course, 'limited > with' is dead in the water... The first sentence may actually be true, but I fail to see the logical connection with the second sentence. Certainly the "type C.T" proposal has exactly the same problem. The type stubs proposal is a bit different because the stub and the completion have different names, but you still have to have both views in your hand and decide which one to use. Here is an example: limited with P; package P_Stub is type Root is tagged separate of P.Root; end P_Stub; with P_Stub; package P is type Root is tagged ...; end P; with P; package Q is type Arr is array (1..10) of P.Root; end Q; with Q; package R is Pain : Q.Arr; end R; with P_Stub; with Q, R; package S is X : P_Stub.Root; -- Illegal, completion not available. Y : Boolean := Q.Arr (1) = Q.Arr (2); -- Legal end S; So on each construct you have to be extra-cautious to determine if you are allowed to peek at the completion or if you must stop at the stub. (There is a similar problem in the current language with private types.) > package P is > type Root is abstract tagged ...; > end P; > > with P; > package Q is > type Win is new Q.Root with ... > end Q; > > with Q; > package R is > Main : Q.Win; > end R; > > limited with P; > with Q, R; > package S is > type Any_Root is access all P.Root; -- Legal? > Child : Q.Win := R.Main; > end S; I have actually played with a different mechanism to solve this conundrum. I am very far from having RM words at this point, but the idea would be that P.Root would become incomplete again in the scope of the "limited with P". This means that simply by looking at the context clause (and the ones you inherits) you could decide which view of P needs to be made available. There would not be ripple effects, because it would all be based on the scope of with clauses. With this mechanism, the declaration of Any_Root would of course be legal, but the declaration of Child would not, because Q.Win would be a type with an incomplete component. This might seem a bit odd, but it's probably OK for the user: if you want to declare a variable of type Q.Win, then surely you can write a "with P" so the "limited with P" appears pointless. I realize that this is all half-baked... ************************************************************** From: Tucker Taft Sent: Wednesday, March 26, 2003 9:46 AM > I realize that this is all half-baked... You should also include in every example a visible non-library-unit renaming of P, just to be sure you have a good answer for that. ************************************************************** From: Randy Brukardt Sent: Wednesday, March 26, 2003 1:29 PM > The first sentence may actually be true, but I fail to see the logical > connection with the second sentence. Certainly the "type C.T" proposal > has exactly the same problem. type C.T does not meet the SigAda criteria IMO, and I'm thinking about it as little as possible. I'd motion to kill it, but I'll have to wait until June to do that. ... > So on each construct you have to be extra-cautious to determine if you > are allowed to peek at the completion or if you must stop at the stub. > (There is a similar problem in the current language with private types.) Of course, every Ada compiler has to be prepared to handle two views of the same type. I'm concerned about two views of the same (original) fully expanded name, and especially two views of the CONTENTS of the same name. In Janus/Ada, we handle visibility of names by two mechanisms: standard scoping, and a visibility bit. The second was added to handle things that are invisible but still are in scope: units in the semantic closure, inherited private record components, etc. Package renames are implemented by a pointer to the original package node. But neither of these allow a name to be both visible by one path and invisible by another path. If we have a path that allows us access to all the package contents, then any path that will get us to the package node is going to have access to all the package contents. Similarly, the limited with proposal doesn't bring discriminants in. These of course are part of the contents of the type. If the type has discriminants, we don't have a way to prevent accessing them (or knowing about their existence) if some other view supposedly doesn't have them. (Some such uses would be illegal for other reasons, but this certainly comes up if we allow Tucker's deferencing rules; but I'd expect that it would come up in other ways as well.) > I have actually played with a different mechanism to solve this > conundrum. I am very far from having RM words at this point, but the > idea would be that P.Root would become incomplete again in the scope of > the "limited with P". This means that simply by looking at the context > clause (and the ones you inherits) you could decide which view of P > needs to be made available. There would not be ripple effects, because > it would all be based on the scope of with clauses. There would still be ripple effects in children, but since those happen in the existing language, presumably we can ignore that. (Removing a with from the parent can make children illegal). > With this mechanism, the declaration of Any_Root would of course be > legal, but the declaration of Child would not, because Q.Win would be a > type with an incomplete component. This might seem a bit odd, but it's > probably OK for the user: if you want to declare a variable of type > Q.Win, then surely you can write a "with P" so the "limited with P" > appears pointless. Well, that's counter to the principle of least visibility (that is, only use the least visibility that you have to get the job done). That effectively makes limited with look like a hack, because you can only use it in very limited circumstances, and it possibly would break other, unrelated code. Of course, it IS a hack; I'm certain that an elegant solution to this problem doesn't exist. In any case, this sounds like no real proposal is going to be on the table soon, so I'm going to give this whole subject lower priority for now. ************************************************************** From: Pascal Leroy Sent: Thursday, March 27, 2003 4:54 AM Well, I admit that it may be a bit surprising to the user, but not more than ripple effects or that an unused package renaming in an auxiliary unit affecting the visibility. I contend that it's not going to happen often in practice. And it's probably not worse (and less frequent) than name clashes in use clauses or reemergence in generics, which when you think of it are odd too (only, we've got used to them). > package P is > type Root is abstract tagged ...; > end P; > > with P; > package Q is > type Win is new Root with ... > end Q; > > with Q; > package R is > Main : Q.Win; > end R; > > limited with P; > with Q, R; > package S is > type Any_Root is access all P.Root; -- Legal? > Child : Q.Win := R.Main; > end S; Look at your original example again. Why the hell do you write "limited with P" in S? I can see two reasons: 1 - S and P are part of a cycle. Then P will need to have a "limited with S" and I'm not sure why you wouldn't write "with P" in S. At any rate, this looks like cheesy OO design to me. You'll have to give a more convincing example to demonstrate that there is a usability problem here. 2 - You want to document/ensure that S only uses the limited view of P. Then I am going to argue that my model is exactly what you want, because it effectively hides the full view of P. ************************************************************** From: Pascal Leroy Sent: Thursday, March 27, 2003 5:18 AM > I would encourage > Pascal to produce something soon, even if it lacks > detailed wording, so long as it captures his intent > in a way that others can review it. OK, you'll find below an attempt at capturing my ideas on visibility in RM wording. I'm pretty sure there will be adjustments needed (I never seem to get wording right) but it should give enough detail for others (in particular, Tuck and Randy) to review. This approach is supposed to: 1 - Solve the two-view problem: at the end of the context clause the compiler knows which view it must grab. 2 - Solve the ripple effect problem: it's all phrased in terms of the scope of with clauses, so distant changes in visibility/renamings don't affect the legality of a unit. Assuming that these rules work, I would be strongly in favor of using them for the "type C.T" proposal, as it has the same problems. The current rules of AI 217-07 are nearly impossible to implement for an incremental compiler. (Well, I would be strongly in favor of killing AI 217-07 altogether, but that's a different discussion.) Let me look specifically at two examples to explain the consequences of the rules. First, an example from the minutes of the Padua meeting: package Q is type T is ...; Z : T; end Q; limited with Q; package P is type Acc is access Q.T; X : Acc; end P; with Q; package A is package MyQ renames Q; end A; limited with Q; with P; with A; package R is P.X.all; -- Illegal, causes freezing of the incomplete type Q.T. A.MyQ.Z -- Illegal, full view of Q hidden from all visibility. Q.Z -- Illegal, full view of Q hidden from all visibility. end R; And then the example that Randy circulated recently: package P is type Root is abstract tagged ...; end P; with P; package Q is type Win is new Root with ... end Q; with Q; package R is Main : Q.Win; end R; limited with P; with Q, R; package S is type Any_Root is access all P.Root; -- Legal. Child : Q.Win := R.Main; -- Illegal, freezing P.Root. end S; The rules will be found below, each paragraph of wording following by a [bracketed] paragraph explaining the purpose of the rule. -- - Add after 10.1.2(5): A with_clause is *more nested than* another with_clause if they are both part of the same context_clause, or if the scope of the first with_clause is entirely included in the scope of the second with_clause. [We need to define the nesting of with_clauses to phrase a number of rules that describe the interaction between nonlimited_ and limited_with_clauses appearing on different units. Note that a with_clause is more nested than itself (not that it matters much, but it make some of the wording simpler).] - Replace 10.1.2(6) by: A library_item is *explicitly mentioned* in a with_clause if it is denoted by a library_unit_name in the with_clause. A library_item is mentioned in a nonlimited_with_clause if it is explicitly mentioned or if it is denoted by a prefix in the with_clause. A library_item is mentioned in a limited_with_clause if it is explicitly mentioned or if it is denoted by a prefix in the with_clause, and that with_clause is not more nested than a nonlimited_with_clause that mentions the same library_item. [For nonlimited_with_clauses, this doesn't change the current RM. For limited_with_clause, it says that if for instance the specification of Q has a "with P" and the body of Q has a "limited with P.R.S" then P is not mentioned in the limited_with_clause. So the limited_with_clause has no effect for P; in particular it doesn't revert to the limited view of P. On the other hand, it makes the limited views of P.R and P.R.S visible. This is mostly a methodological restriction, as we have (pathological) cases where the limited view hides the full view. However, we want to make such cases infrequent, in part to avoid confusing users, and in part to account for the fact that the checks associated with "reverting" to a limited view may be a bit costly.] - Add after 10.1.2(8): A limited_with_clause which explicitly mentions a library_item shall not be more nested than a nonlimited_with_clause which mentions the same library_item. [Thus, if for instance the specification of Q has a "with P.R", it shall not have a "limited with P" or a "limited P.R". The same applies to the body of Q, its subunits, etc. On the other hand, a "limited with P.R.S" is fine. Again this is to avoid reverting to the limited view as much as possible.] - Add after 8.3(20): The full view of a library_item is hidden from all visibility within the scope of a limited_with_clause that mentions it. The limited view of a library_item is hidden from all visibility within the declarative region of the library_item, and within the scope of a nonlimited_with_clause that mentions it. [The first sentence is the rule that prevents ripple effects and the two-view problem. If you are is the scope of a "limited with P", the full view of P is hidden from all visibility, even if P would otherwise be visible indirectly (e.g. by a renaming in an auxiliary package). So the limited_with_clause effectively reverts to the limited view of P, and the types it declares are incomplete, and therefore subject to the limitations described in 3.10.1. From an implementation standpoint, it means that compilers can decide early (i.e., after looking at the context_clauses of a unit and of its ancestry) what view of P needs to be made available, and they never have to juggle with both views. Note that I expect this rule to kick in only in rather pathological situations: if you are at a place where the full view of P is visible, it is unclear why you would want to write "limited with P", except perhaps to document the fact that at this point you only use the limited view.] - Replace 3.11.1(8) by: A type is completely defined at a place that is after its full type definition (if it has one) and after all of its subcomponent types are completely defined. Furthermore, if the type is declared within the visible part of a library_item, it is *not* completely defined within the scope of a limited_with_clause that mentions the enclosing library_item. A construct that causes freezing of a type shall occur at a place where the type is completely defined (see 13.14 and 7.3). [Within the scope of a "limited with P", not only is P.T an incomplete type, but any type that has a part of type P.T is incompletely defined. Such types could exist because they would be declared in auxiliary units where the full view of P is visible. This rule makes it possible to piggyback on the freezing rules: within the scope of a "limited with P", any construct that would cause freezing of P.T is illegal. Surely compilers already have mechanisms to implement the freezing rules, and for instance on every object_declaration they have to verify that the type doesn't have incompletely defined subcomponents. It is hoped that it would be relatively straightforward to modify these mechanisms to implement this rule. One difference though is that currently compilers never have to look at the parent part because 13.14(7) ensures that it is frozen. To implement the new rule, the parent part would have to be inspected. If performance is a concern, this part of the check could only be performed in the scope of a limited_with_clause.] - Replace 13.14(17) by: A construct that causes freezing of a type shall occur at a place where the type is completely defined. [A simple copy of the last sentence of 3.11.1(8). This rule is redundant anyway. Note that within the scope of a "limited with P" the type P.T is incompletely defined, but it is still frozen as explained in the second sentence of 13.14(2) (I don't want to mess with the freezing rules). What we have here is a construct that causes freezing occurring after the freezing point, and such a construct is illegal if the type has reverted to "incompletely defined".] ************************************************************** From: Tucker Taft Sent: Thursday, March 27, 2003 5:49 AM Looks good to me! Thanks for making the effort to provide this sooner rather than later. I'll see if I can figure out how an approach like this could apply to the incomplete child type proposal, and I presume Randy will do the same for the type stub proposal. ************************************************************** From: Tucker Taft Sent: Thursday, March 27, 2003 9:11 AM I have interspersed some detailed comments with your proposed wording. But first, to summarize... My major comment is that we should restrict ourselves to talking about what is in the limited view of a package, namely it contains limited views of nested packages, and incomplete type declarations. It seems unnecessary to go and render incomplete other uses of a type declared in the package, presuming those uses appeared in places where a full view of the package existed (e.g. objects of the type, or subtypes of the type, or types derived from subtypes of the type, or ...). This is mostly to simplify implementations, though I think it also reduces the number of wording changes. Right now, the freezing rules never need to be of concern when handling statements. They only apply to expressions and names that appear in declarations. With the change you are proposing, freezing checks have to be performed in all statements as well, and that seems like overkill. I think the very *good* part about your proposal is that it implies that any renaming of a package, when within the limited view of the package, is considered limited. I think that is the big improvement over earlier proposals, and is probably the most intuitive for the user. If we focus just on the limited package view and what it contains, and not worry about visible entities that are somehow based on information gleaned from a full view of the package, then that keeps the implementation effort focused on package names, and whether they denote a full view or an incomplete view. That is a nice place to focus, because there is already work needed when dealing with package names and renames, to be sure to include children, decide whether the private part is visible, etc. This might be a relatively small increment on that. I fear if you start burdening every object/subtype name usage with worries about limited views, you will significantly increase the implementation burden. ... > - Replace 10.1.2(6) by: > > A library_item is *explicitly mentioned* in a with_clause if it is > denoted by a library_unit_name in the with_clause. A library_item is > mentioned in a nonlimited_with_clause if it is explicitly mentioned or > if it is denoted by a prefix in the with_clause. A library_item is > mentioned in a limited_with_clause if it is explicitly mentioned or if > it is denoted by a prefix in the with_clause, and that with_clause is > not more nested than a nonlimited_with_clause that mentions the same > library_item. > > [For nonlimited_with_clauses, this doesn't change the current RM. For > limited_with_clause, it says that if for instance the specification of Q > has a "with P" and the body of Q has a "limited with P.R.S" then P is > not mentioned in the limited_with_clause. So the limited_with_clause has > no effect for P; in particular it doesn't revert to the limited view of > P. On the other hand, it makes the limited views of P.R and P.R.S > visible. This is mostly a methodological restriction, as we have > (pathological) cases where the limited view hides the full view. > However, we want to make such cases infrequent, in part to avoid > confusing users, and in part to account for the fact that the checks > associated with "reverting" to a limited view may be a bit costly.] I think we might be able to simplify this by talking about the scope of a limited view of a package, and say that it is the scope of a limited with clause that mentions it minus the scope of any more nested (nonlimited) with clause that mentions it, or any renaming of it. I presume we are still disallowing a limited with mentioning a library-unit renaming. > - Add after 10.1.2(8): > > A limited_with_clause which explicitly mentions a library_item shall not > be more nested than a nonlimited_with_clause which mentions the same > library_item. I think this is approximately equivalent to saying that a limited with clause is illegal if it has no effect (because it is immediately canceled by some outer or same-context-clause nonlimited with). Withs that "skip over" via renamings one of the enclosing packages mentioned in the limited with make this only an approximate equivalence. By the way, I think we may want to disallow a limited with of a package inside a "use" clause for a renaming of the package (or nested package thereof). Or we need to make the "use" lose its effect. In any case, I don't think we want to allow (new) "use" clauses of limited views of packages. ... > - Add after 8.3(20): > > The full view of a library_item is hidden from all visibility within the > scope of a limited_with_clause that mentions it. The limited view of a > library_item is hidden from all visibility within the declarative region > of the library_item, and within the scope of a nonlimited_with_clause > that mentions it. I think you want to talk about library packages here rather than library "items", and make it clear that a non-limited with of a renaming of a library package also hides the limited view. Rather than talking about "hiding" the limited view, we could talk about its scope. We might also want to talk about the scope of the full view instead of using hiding for that. Essentially the scope of the full view is as defined now, except it does not include places that are within the scope of the limited view. The scope of the limited view is defined in terms of the scope of limited/non-limited with clauses that mention it (or a renaming thereof). ... > - Replace 3.11.1(8) by: > > A type is completely defined at a place that is after its full type > definition (if it has one) and after all of its subcomponent types are > completely defined. Furthermore, if the type is declared within the > visible part of a library_item, it is *not* completely defined within > the scope of a limited_with_clause that mentions the enclosing > library_item. A construct that causes freezing of a type shall occur at > a place where the type is completely defined (see 13.14 and 7.3). This is where I think you are getting more complicated than necessary. Rather than talking about whether a type is completely defined, I would talk only about what P.T denotes, when P is a limited view of a package. I think you are asking for a *much* more complicated implementation effort if you are expecting every use of a type, including in subprogram bodies, to check to see whether it should be considered incomplete. Why not just say that within the limited view of a package P, "P.T", or "Q.T" where Q is any renaming of P, denotes an incomplete view of T. Other subtypes declared using full views of T, or objects declared using full views of T, are not affected. And of course, a limited view of a package only contains nested packages and incomplete type declarations, nothing else. ... > - Replace 13.14(17) by: > > A construct that causes freezing of a type shall occur at a place > where the type is completely defined. Again, I don't think we need to muck with the freezing rules. ************************************************************** From: Pascal Leroy Sent: Thursday, March 27, 2003 10:55 AM > My major comment is that we should restrict ourselves to > talking about what is in the limited view of a package, > namely it contains limited views of nested packages, and > incomplete type declarations. It seems unnecessary to go and > render incomplete other uses of a type declared in > the package, presuming those uses appeared in > places where a full view of the package existed > (e.g. objects of the type, or subtypes of the > type, or types derived from subtypes of the type, > or ...). I am confused here. The problem I am trying to solve is the two-view problem, i.e. the situation where the compiler has to juggle with the limited and the full view of the unit. My fear is that, if that if we don't have the "revert to incomplete" rule, the compiler will end up wandering in the symbol table for the full view of the unit, and we won't have solved the problem. Or maybe this won't happen, i.e. the compiler will never try to look at the full view of the unit? Hmmm, not sure if all the rules of the language combine to prevent this problem from happening... This being said, I must admit that the notion of checking the freezing rules in statement parts is unappealing. ************************************************************** From: Tucker Taft Sent: Thursday, March 27, 2003 2:13 PM If you make all renames of the package show the limited view, and you disallow "use"s, then I don't see why you would be wandering in the package's full-view symbol table at all. But your mileage may vary... > This being said, I must admit that the notion of checking the freezing > rules in statement parts is unappealing. And I think its worse than that. Once we freeze something, we mark it frozen. We would have to ignore that flag and, pretty much on every use, have to check whether the entity is declared somewhere inside a package that has its limited view in scope. Uggh. ************************************************************** From: Randy Brukardt Sent: Thursday, March 27, 2003 6:11 PM I don't see that as particularly bad: it just means more work for the Check_Freezing subprogram. (Or whatever it's called). And if there are places where that's not called, they'd have to be added. I doubt that is anywhere near as much work as implementing limited views themselves. (But, of course, a better idea is welcome...) ************************************************************** From: Pascal Leroy Sent: Friday, March 28, 2003 4:48 AM My opinion exactly. And I doubt that is anywhere near as much work as juggling with two views and having to decide on each and every construct of the language which view you are allowed to access. ************************************************************** From: Pascal Leroy Sent: Thursday, March 27, 2003 3:32 PM > If you make all renames of the package show the limited view, > and you disallow "use"s, then I don't see why you would be wandering > in the package's full-view symbol table at all. But your > mileage may vary... Consider the following example: package P is type T is tagged; function "=" (L, R : T) return Boolean; end P; with P; package Q is type A is array (1..10) of T; X, Y : A; end Q; with Q; limited with P; procedure Main is begin if Q.X = Q.Y then ... end if; end Main; One code generation strategy for the equality in Main is to generate a loop calling P."=" for each component. I fail to see how this can be done without accessing the full view of P. In particular, a clever compiler might just want to use a block comparison if T has no user-defined "=", so it will surely need to look at the full view of P. Also, from the user standpoint, I think that operating on objects (or types) which have incomplete parts is counter-intuitive and can lead to all sorts of surprises. ************************************************************** From: Tucker Taft Sent: Thursday, March 27, 2003 4:58 PM I think we weren't understanding one another. I certainly agree you need to get to the full view of the types declared in P, and all of their "basic" operations, because there might be subtypes, objects, derived types, etc. of them around. I just think you don't need to get to the full view of the package itself. That is, while within the limited view of P, you don't need to do a lookup in the full symbol table of P. Perhaps what you are saying is that you do a *lookup* in P to find the primitive "=" on tagged T needed for creating a composite "="? I guess that surprises me. With this proposal, I think you will have to eliminate any such "implicit" lookups if you do them, and simply store on the type node a reference to its primitive "=" if it is tagged. The explicit operator lookups are fine, since the package cannot be used (and presumably, nor can any types declared in the package) while in the limited view of the package. Hence the explicit operator lookups will never find anything. > Also, from the user standpoint, I think that operating on objects (or types) > which have incomplete parts is counter-intuitive and can lead to all sorts > of surprises. I agree. I don't think this is the way they should or would think about it. The parts don't become limited, just because we are in the limited view of a package where a part's type might have been defined. The key point is that anything already extracted out of the full view of the package is still fine. You just can't extract anything more out of the package, while in its limited view. ************************************************************** From: Randy Brukardt Sent: Thursday, March 27, 2003 6:20 PM Right. You're going to have to have the full view in your symboltable, like it or not. I know that Janus/Ada will fail spectacularly if a package in the semantic closure isn't loaded. (We had a number of bugs like that in the past.) So, there is no option to avoid loading the full views of everything in the semantic closure. But, we can't allow the package view to be full to some users and limited to others. And we can't just use the semantic dependence because of the ripple effects. So I think Tucker's proposal is about right. If the package was in the semantic closure, we'd use a subprogram to go through it and turn off the visibility bits of everything that is not a package or a type. That means that there would be only one lookup view. The other checking would have to be done at the point of use (there wouldn't be anyway to mark it) - presumably the code that determines if we have enough visibility to proceed from the incomplete view to the complete view would have to muck around in a list of with clauses. (I don't think that such a list exists currently.) If the view is limited, we don't allow moving to the complete view. But I'll have to think about this more to be sure that this is doable. ************************************************************** From: Robert Eachus Sent: Thursday, March 27, 2003 5:06 PM I have been reading this ongoing discussion with stunned disbelief. Not that the details are complex, or that some solution is required. It is just that for many of these problems, the simple solution is to make the complicating factors illegal. For example, the issue of renaming packages which contains potentially incomplete types. The simple solution is to make it illegal to rename a package if the context clause doesn't contain the completion of any incomplete types in that package. The problem we are trying to solve could be solved by allowing implementations to require both the unit with the incomplete type and its completion to be compiled together. I think that is a bit draconian, but I also think that some of the problem cases being discussed are a result of trying to make the rules for these types too orthogonal. I can live with some pretty severe restrictions to make such declarations available. Not renaming packages with an incomplete view of a type is pretty mild. ************************************************************** From: Randy Brukardt Sent: Thursday, March 27, 2003 6:29 PM > It is just that for many of these problems, the simple solution is to > make the complicating factors illegal. No, we can't do that, because that is precisely the ripple effect that we're trying to avoid: adding or removing a with clause on a distant unit changing the legality of a unit. > For example, the issue of renaming packages which contains potentially > incomplete types. The simple solution is to make it illegal to rename a > package if the context clause doesn't contain the completion of any > incomplete types in that package. That's not the issue at all. Look at the examples that Pascal posted more closely. The package which was renamed was a full view. The incomplete view is coming from the local package; the full view is coming from the semantic closure. The point of that example is that visibility isn't the right model, because with a renaming, we can have visibility onto the full view -- and thus even using a visibility model causes ripple effects. > The problem we are trying to solve could be solved by allowing > implementations to require both the unit with the incomplete type and > its completion to be compiled together. I think that is a bit > draconian, but I also think that some of the problem cases being > discussed are a result of trying to make the rules for these > types too orthogonal. That's similar to what alternative 05 does for type stubs, but is much stronger. And I don't think it helps, but it makes it a lot harder to use these things (particularly if you want to avoid extra packages). Besides, "allowing" implementations to do something is not likely to be helpful if the implementation has no way to do anything with that (that's certainly true of Janus/Ada, which can only process one unit at a time, and I think it is true of Gnat as well). ************************************************************** From: Robert Eachus Sent: Thursday, March 27, 2003 8:44 PM I think you are taking what I was saying a little too literally. If it takes subtle analysis do decide when a renaming needs to be outlawed because it causes visibility clashes, fine. All the user needs to know is that a renaming of a package with an incomplete type may be illegal in some contexts. That won't bother many, if any, users. (Most package renamings I see are either library renamings or abbreviations of predefined language units.) ************************************************************** From: Tucker Taft Sent: Thursday, March 27, 2003 9:21 PM Robert, You are definitely confused about the issue. We have already agreed to disallow renaming of limited views. What we are dealing with now are renamings that were done in a place which only saw the full view, and is totally unaware that a "limited with" even exists. The problem is that a unit that has a limited-with might also "with", directly or indirectly, something that already has a rename of the full view. This means that it can get at this renaming. What should it see if it tries to use the renaming. Should it see the full view, which was what existed when the renaming was declared, or should it see the limited view, which is all it can see by naming the package directly? In Pascal's proposal, you will see the limited view, even though when the renaming was declared, it was a renaming of the full view. I think Pascal's proposal is the right answer here. It gibes with what happens when you go through a renaming of a library unit with children. The children you see are based on those that are visible at the point of reference to the renaming, rather than at the point where the renaming was declared. ************************************************************** From: Robert Dewar Sent: Thursday, March 27, 2003 10:12 PM > That's similar to what alternative 05 does for type stubs, but is much > stronger. And I don't think it helps, but it makes it a lot harder to use > these things (particularly if you want to avoid extra packages). Besides, > "allowing" implementations to do something is not likely to be helpful if > the implementation has no way to do anything with that (that's certainly > true of Janus/Ada, which can only process one unit at a time, and I think it > is true of Gnat as well). Actually it is not true of GNAT at all, it is perfectly reasonable for GNAT to process multiple units in the sense of requiring some units to be around when another unit is compiled. ************************************************************** From: Robert Eachus Sent: Thursday, March 27, 2003 11:09 PM Exactly the situation I thought I was addressing. ;-) Pascal's approach of seeing the limited view is troubling, but I think we can outlaw the case without much pain. Yes, it means making it illegal for a unit with a limited view of a type to also have a full view of the type. The problem case would seem to be a package that created many types completed in several other units. This would say that in that case you couldn't use the completed versions of some types in the bodies of packages that completed other types. This would occasionally require multiple child packages, some of which completed types, others that created objects of those types. Shrug. If we are trying to compare several approaches on the assumption that they will cause minimal implementation pain, I think we should be willing to accept such limitations and see where it leads. If one of the three approaches looks much better given such limitations, great. Then we can choose that approach, knowing that it is a workable solution and see how much we can extend it before the implementation difficulties become too great. That is what I was driving at. The current approach seems to be to have full implementation designs for the most general version of each approach, and I think that that way lies madness. In this sense I think Robert Dewar's recent message is a breath of fresh air. GNAT might or might not "take advantage" of some permitted implementation freedom. That doesn't mean we should reject the potential restriction ahead of time. If allowing the restriction now makes a particular approach acceptable, we can either argue the politics later, or accept the restriction for now, and look into removing it later. An implementation restriction on a feature that currently doesn't exist is not a hardship for anyone. ************************************************************** From: Tucker Taft Sent: Thursday, March 27, 2003 9:37 PM Well I haven't seen a real answer to my suggestion: Being in the limited view of a package should have no effect on objects or subtypes declared using the full view. The only effect should be on what you get if you (explicitly) look up P.T, or renaming-of-P.T, and what you get is an incomplete view of T. But there might be full views of T already embedded in objects or subtypes or derived types or composite types, and none of those are affected. If implicit lookups in P are needed to support these preexisting objects/subtypes/derived types, then those implicit lookups should use the full view, but these should be rare, and could probably be handled in some other way (e.g., hanging the primitive "=" directly off the type, rather than having to look it up each time it is needed to implement the equality of an enclosing composite object). I think this eliminates any need to futz/muck/mess with freezing rules, and localizes the changes to explicit package lookups. This is reminiscent of what we already do for library packages. We have the original view of a package, and then we construct an "augmented" view when a package is "with"ed, and in the augment view we add children that get "with"ed. The original view is preserved in a read-only state. The augmented view is built up incrementally during context clause processing. I could see also having a "limited" view for a package. It could be constructed based on the full view if available, or by doing a "loose parse" of the source of the package. This limited view would be used for all explicit lookups in the package when the limited view is in scope due to mention in a limited with clause (and not "canceled" due to mention in an inner non-limited "with"). ************************************************************** From: Pascal Leroy Sent: Friday, March 28, 2003 4:47 AM First, let me make clear that from my perspective, the Ripple Effect Problem is way more severe than the Two-View Problem. The former is a no-no for me, as it kills incremental compilation. The latter is a nuisance, but if I absolutely have to, I suppose I can live with it. So my objection to your position, Tucker, is that, if you don't have the "revert to incomplete" rule, the language design looks botched. For instance: package P is type T is ...; end P; with P; package Q is type T is record C : P.T; end record; end Q; limited with P; with Q; package Main is X : P.T; -- Illegal Y : Q.T; -- Legal end Main; The fact that the declaration of X is illegal but the declaration of Y is legal looks like a wart to me. Try to explain this to real users! It would seem much more natural to say that they are both illegal. As I understand it, you main objection is one of implementation complexity (I don't buy arguments based on the number of words in the RM that have to change). Since all incarnations of AI 217 will require a lot of work anyway, I cannot get too excited. It seems to me that we should concentrate on finding a nice and clean language mechanism, even if that's at the expense of some implementation complexity. > I think we weren't understanding one another. I certainly > agree you need to get to the full view of the types declared > in P, and all of their "basic" operations, because there > might be subtypes, objects, derived types, etc. of them around. > > I just think you don't need to get to the full view of the > package itself. That is, while within the limited view of P, > you don't need to do a lookup in the full symbol table of P. > > Perhaps what you are saying is that you do a *lookup* in P to > find the primitive "=" on tagged T needed for creating a composite > "="? I guess that surprises me. After checking the code, it turns out that we don't in this case. But there are other situations where we do. For instance: package P is type T is ...; function F return T; end P; with P; package Q is subtype S is P.T; end Q; limited with P; with Q; package Main is type NT is new Q.S; end Main; In order to build the inherited subprograms for NT, we scan the declarative part where T is declared, i.e. the full view of P. And we are certainly not going to change this, it would be way too disruptive. If we had to implement the rules that you propose, we would have to live with the Two-View Problem. But again, my objection is mostly one of language design. > And I think its worse than that. Once we freeze something, we > mark it frozen. We would have to ignore that flag and, pretty > much on every use, have to check whether the entity is declared > somewhere inside a package that has its limited view in scope. I realize that implementation complexity depends a lot on how things are structured. However I am a bit surprised by your claim, because for us changing the freezing rules to deal with type "reverting to incomplete" would literally be an afternoon project. Let me give a few implementation notes to make the discussion more concrete. When a type gets frozen we do mark it with a bit, Sm_Frozen, in the Diana tree. It turns out that we run the freezing checks on statement parts as well as declarative parts, because some constructs in statement parts declare anonymous subtypes that need to get frozen (don't ask). Most of the time this processing ends very quickly because it finds that Sm_Frozen is set. But I realize that this is an idiosyncrasy of our implementation, and that most compilers probably don't look at statement parts. To implements the "revert to incomplete" rule, I would probably have a global bit indicating whether or not I am in the scope of a limited with. (I could be more clever and only set this bit when I am in the scope of a "limited with P" and P is in the closure through some other path, but I guess that I would be too lazy to do that.) During the freezing checks, this global bit would cause the setting of Sm_Frozen to be ignored, so we would effectively traverse the subcomponents, etc. of each type. Once a type has been found to be OK (i.e., to not have any incomplete subcomponent) I would mark it with a temporary bit so as to avoid redoing the traversal over and over again. This means that every type would be analyzed (at most) once, so the performance impact should be minimum. Obviously there would be no performance impact at all when you are not in the scope of a limited with. This doesn't sound like rocket science to me. Could you maybe explain your implementation concerns in a bit more detail? ************************************************************** From: Pascal Leroy Sent: Friday, March 28, 2003 4:54 AM > Well I haven't seen a real answer to my suggestion: > Being in the limited view of a package should have no > effect on objects or subtypes declared using the > full view. The only effect should be on what you > get if you (explicitly) look up P.T, or renaming-of-P.T, and what > you get is an incomplete view of T. But there might > be full views of T already embedded in objects or > subtypes or derived types or composite types, and > none of those are affected. If you want my "real answer" to this it is that this approach really looks like a hack and is going to puzzle users. ************************************************************** From: Tucker Taft Sent: Friday, March 28, 2003 8:28 AM > So my objection to your position, Tucker, is that, if you don't have the > "revert to incomplete" rule, the language design looks botched. I don't agree, but in any case, we should agree we are talking about corner cases here, because this only occurs when you are in the scope of a limited view, but the full view is in your semantic closure. Hence, I believe as few words and as little implementation complexity should be devoted to this area as possible, subject to eliminating ripple effects. I believe the key thing is to limit ourselves to talking about limited views of *packages*, and not get hung up on imposing limited views on anything else. Packages are already a bit funny because of public/private children and visible/private parts. > For > instance: > > package P is > type T is ...; > end P; > > with P; > package Q is > type T is > record > C : P.T; > end record; > end Q; > > limited with P; > with Q; > package Main is > X : P.T; -- Illegal > Y : Q.T; -- Legal > end Main; > > The fact that the declaration of X is illegal but the declaration of Y > is legal looks like a wart to me. It doesn't look like a wart to me, because the types used to implement Q.T are the business of Q, and should not be relevant to a user of Q. Suppose Q.T were implemented in terms of U.T, which is implemented in terms of V.T, which is implemented in terms of W.T, which just happens to have a teeny little component in one of its many variant parts that happens to be of a type that is derived from P.T? Or suppose we change Q to: with P; package Q is type R is private; ... private type R is record C : P.T; end record; end Q; Clearly with your defender-of-privacy hat on, you wouldn't want the characteristics of R to be affected when a user of Q happens to have a limited view of P. > ... Try to explain this to real users! The key is that the compilation unit has a limited view of *P*, so it can only see certain things in P, and it sees only some features of those via an expanded name involving P (or one of its renamings). It doesn't affect any other compilation unit, nor any names that don't involve P or one of its renamings. > It would seem much more natural to say that they are both illegal. I don't agree, and I really think this way lays big trouble. Suppose you instantiate a generic, which instantiates a generic, which declares an object of a derived type of P.T. I really think you are recreating a ripple effect, where some distant package's decision to use something from P affects whether you can instantiate it. > > As I understand it, you main objection is one of implementation > complexity (I don't buy arguments based on the number of words in the RM > that have to change). I agree that number of words is not a great guage, but it is relevant, and especially when you are affecting the legality of essentially every construct in the language. Would it be legal to call a subprogram that had a default expression that happened to use a 'Access attribute of an object of type P.T? Or how about use a record type where one of the components had a default expression that referenced the identity attribute of an exception declared in P? > ... Since all incarnations of AI 217 will require a > lot of work anyway, I cannot get too excited. It seems to me that we > should concentrate on finding a nice and clean language mechanism, even > if that's at the expense of some implementation complexity. Well, I believe your proposal is much cleaner if you focus strictly on what you can see when you explicitly try to refer to components of a limited view of a package via an expanded name. You are opening an unnecessary can of very nasty worms if you start changing the characteristics of things in other compilation units, other than renamings of this same package. ... > In order to build the inherited subprograms for NT, we scan the > declarative part where T is declared, i.e. the full view of P. And we > are certainly not going to change this, it would be way too disruptive. > If we had to implement the rules that you propose, we would have to live > with the Two-View Problem. But again, my objection is mostly one of > language design. I guess we have a list of primitives hanging off every type, so we never have to scan the package itself, but I agree, deriving is probably tricky. But again, what happens if the type you are deriving from is a private extension of some ancestor of P.T, but when you get into the private part, you discover its immediate parent is P.T itself. I really think this shouldn't affect legality of the new type derivation. >>And I think its worse than that. Once we freeze something, we >>mark it frozen. We would have to ignore that flag and, pretty >>much on every use, have to check whether the entity is declared >>somewhere inside a package that has its limited view in scope. > > > I realize that implementation complexity depends a lot on how things are > structured. However I am a bit surprised by your claim, because for us > changing the freezing rules to deal with type "reverting to incomplete" > would literally be an afternoon project. Let me give a few > implementation notes to make the discussion more concrete. > ... > This doesn't sound like rocket science to me. Could you maybe explain > your implementation concerns in a bit more detail? I don't want to focus the argument on our implementation complexity, because I think the argument can be better focused on areas like privacy and ripple effects which I know are near-and-dear to your heart. But in general, I think making this proposal depend subtly on the freezing rules, and what is frozen by what sorts of uses, is a mistake, and completely unnecessary to achieve all of your objectives. As I said in my first paragraph, we are talking about corner cases, and they don't deserve the effort that will be involved in defining exactly what is legal when using entities that in some vague way depend on the full view of P. Renamings of P are clear, everything else just gets muddier and muddier in my view. ************************************************************** From: Pascal Leroy Sent: Friday, March 28, 2003 8:48 AM > I don't agree, but in any case, we should agree we > are talking about corner cases here, because this only > occurs when you are in the scope of a limited view, but > the full view is in your semantic closure. I definitely agree that this is a corner case. I still think it's important to come up with a model that is not too surprising to users, and consistent with the rest of the language. But if we can't, I'm not going to throw the baby with the bathwater. > Or suppose we change Q to: > > with P; > package Q is > type R is private; > ... > private > type R is record > C : P.T; > end record; > end Q; > > Clearly with your defender-of-privacy hat on, > you wouldn't want the characteristics of R to > be affected when a user of Q happens to have > a limited view of P. That's a rather compelling argument, and I need to give it more thought over the week-end. Of course, my immediate answer would be to say that you have to assume-the-worst for private types, but I'm not sure I like that, because it would disallow perfectly reasonable usages of perfectly good private types. > I don't want to focus the argument on our implementation > complexity, because I think the argument can be better > focused on areas like privacy and ripple effects which > I know are near-and-dear to your heart. Right, language design, not implementation complexity, should be the focus here. ************************************************************** From: Randy Brukardt Sent: Friday, March 28, 2003 7:24 PM > I guess we have a list of primitives hanging off every type, so > we never have to scan the package itself, but I agree, deriving > is probably tricky. But again, what happens if the type you > are deriving from is a private extension of some ancestor of > P.T, but when you get into the private part, you discover its > immediate parent is P.T itself. I really think this shouldn't > affect legality of the new type derivation. Yuck! This example gives me heartburn, precisely because we do exactly what Rational does. It MIGHT work anyway (because I think that the visibility bits are ignored for the purposes of derivation - but I'm not at all sure about that) -- that's a very fragile area, and I hate the idea of having to muck with it because of some completely unrelated feature. I guess what worries me here is that we're going to end up with two views in all cases except those that cause ripple effects -- which is hardly going to help implementation or usability. But I don't see a showstopper here. And I certainly agree that this is a corner case; having it perfect isn't critical. (The same can be said about dereferencing rules, BTW, because the only cases that we'd be worrying about are the same ones as here: where we're in the semantic closure of P but only have a limited view. For type stubs, that's where we're in the semantic closure of the completing type, but only have visibility on the incomplete type. Neither sounds real likely.) ************************************************************** From: Dan Eilers Sent: Friday, March 28, 2003 11:10 AM > The fact that the declaration of X is illegal but the declaration of Y > is legal looks like a wart to me. Try to explain this to real users! > It would seem much more natural to say that they are both illegal. This looks easy to explain to real users by citing the rule that adding a "with" clause to a unit can't make anything in the unit illegal that was previously legal. The rule is just being extended to "limited with" clauses. ************************************************************** From: Tucker Taft Sent: Friday, March 28, 2003 11:42 AM I agree with this principle, and we should try to preserve it, though it isn't quite true for "with" clauses, and it won't be true with limited with if you already refer to a renaming of the package (which is admittedly a real corner case). But I agree that adding a limited with for package P should not affect the legality of anything in the compilation unit other than existing references to package P or renamings thereof (or nested packages thereof). It isn't quite true for "with" clauses because adding a "with" clause can hide something that was "use" visible. E.g.: package P1 is type P2 is (tbd); end P1; with P1; use P1; with P2; -- Makes decl of X illegal procedure Main is X : P2; -- becomes illegal begin null end; But if the name "P2" doesn't appear in the compilation unit, then adding a "with" clause for P2 cannot make anything illegal. ************************************************************** From: Robert I. Eachus Sent: Friday, March 28, 2003 7:37 PM > I agree with this principle, and we should try to preserve it, though > it isn't quite true for "with" clauses, and it won't be true with limited > with if you already refer to a renaming of the package (which is admittedly > a real corner case). But I agree that adding a limited with for package P should > not affect the legality of anything in the compilation unit other > than existing references to package P or renamings thereof (or nested packages thereof). I can't believe I can see this so clearly, and the everyone else keeps looking at the trees. Let me try to explain my logic, and see if you can punch holes in it. Something is wrong, and I suspect that I am just not communicating my point clearly enough.... Thesis: It should be illegal for a context to include a "limited with P;" clause if the remainder of the context semantically depends on P. In other words, if "limited with P;" doesn't add visibility while preventing a semantic dependence on P, someone is making a mistake. The whole idea of "limited with" is that a unit with a "limited with P;" does not semantically depend on P. If a unit semantically depends on P for some other direct or indirect reason, then a "limited with P;" is at best a waste of breath. I argue that we should just make it illegal. There is an error somewhere. We don't know where. But trying to make complex rules and add lots of compiler support for something which is not just a corner case but ALWAYS an error is silly. The same type of logic holds in the alternative proposals. In every case we should remember what the goal is, and not try to add bells and whistles that are of no possible use. In the child package approach, we want to create child packages that do not semantically depend on their parents. Cases where a child both does and does not semantically depend on the parent (directly or indirectly) are both logically impossible and silly. ************************************************************** From: Randy Brukardt Sent: Friday, March 28, 2003 8:01 PM > In other words, if "limited with P;" doesn't add visibility while > preventing a semantic dependence on P, someone is making a mistake. Yes, of course. But we can't prevent that. > The whole idea of "limited with" is that a unit with a "limited with P;" > does not semantically depend on P. If a unit semantically depends on P > for some other direct or indirect reason, then a "limited with P;" is at > best a waste of breath. I argue that we should just make it illegal. > There is an error somewhere. We don't know where. But trying to make > complex rules and add lots of compiler support for something which is > not just a corner case but ALWAYS an error is silly. We tried that. But that is a severe ripple effect, and such an effect kills incremental compilation -- which means that Rational cannot support such a feature in their Ada compiler. We can't be putting vendors out of the Ada business in this Amendment! We don't have enough as it is. In case you don't see the problem, consider the program: package P is type T is tagged ... type Index is ... end P; package Q is procedure Proc ... end Q; with Q; limited with P; package R is type Acc is access all P.T'Class; end R; We all agree this is legal. Now, imagine that during maintenance, someone decides that they need a version of Proc that takes an Index. So, they change Q to: with P; package Q is procedure Proc ... procedure Proc (I : in P.Index; ...); end Q; Now, by your rule, R is now illegal. That's the ripple effect; for a compiler like Rational's, every compilation unit that might semantically depend on Q has to be checked. In real systems, that's pretty much everything. Note that Q doesn't depend on P.T at all, it just happens to depend on something that's coincidentally in P. That tends to suggest that Tucker is right in the discussion he and Pascal are having. Indeed, it seems that there would be a ripple effect with Pascal's rule: if someone adds an Index component to some type in Q, that would make uses of that type illegal in R. I can't imagine that's what Pascal wants. package Q1 is type Priv is private; private type Priv is record A : Integer; end record; end Q1; with Q1; limited with P; package R1 is type Acc is access all P.T'Class; Obj : Q1.Priv; end R1; All is OK here. Now, if a maintenance programmer changes Q1 to get better type checking: with P; package Q1 is type Priv is private; private type Priv is record A : P.Index; end record; end Q1; with Q1; limited with P; package R1 is type Acc is access all P.T'Class; Obj : Q1.Priv; -- Illegal by Pascal's proposed rules. end R1; That looks like a second-hand ripple effect to me. It's especially annoying because P.Index is never used as an incomplete type. (The big possible annoyance with limited with is the lack of control over what becomes incomplete. Here we see that it can be too much.) ************************************************************** From: Robert I. Eachus Sent: Friday, March 28, 2003 8:42 PM > We tried that. But that is a severe ripple effect, and such an effect kills > incremental compilation -- which means that Rational cannot support such a > feature in their Ada compiler. We can't be putting vendors out of the Ada > business in this Amendment! We don't have enough as it is. Now I see what you are worried about, but I still don't see it as specific to this feature. If instead, package Q was changed so that Proc required a new parameter, then any unit that depended on Q might become illegal. Sure this new rule would means that modifying any unit that R depends on by adding a with P will make R illegal, and that would mean adding a new check. But I just don't see any better solution. In reality, as I said we will see very few cases where a unit with a limited with will be in some metric "distant" from the unit it does't depend on. ;-) If we have to come up with some restrictive rule that makes it easier for Rational, lets do that. But I think that trying to make the dual path work is a waste of effort. Let me take your example and extend it a bit... with R; -- added package P is type T is tagged record X: R.Acc; -- added ... end record; type Index is ... end P; package Q is procedure Proc ... end Q; with Q; limited with P; package R is type Acc is access all P.T'Class; end R; Still legal right? Now add your with P to package Q: with P; package Q is procedure Proc ... procedure Proc (I : in P.Index; ...); end Q; and there is NO EASY WAY to make the program legal. It is not a minor detail, it is a major design problem. And this will be the normal case. The only reason to have limited with is to break cycles. If a "limited with P;" can be added to Q, then you are okay. So my understanding is that this example is a red herring. The real ripple effect is that you can't add a with P; you have to add a limited with P; to Q if you want things to compile. ************************************************************** From: Robert I. Eachus Sent: Friday, March 28, 2003 11:52 PM I did some more thinking about this example and mapped it to the "classic" example using employees and offices. The surprise is that adding a dual path does not make the program illegal--at least without my proposed rule--but the "harmless" maintenance change does even more violence to the program structure: limited with Employees; package Offices is type Office is tagged private; type Office_Pointer is access all Office'Class; ... private type Office is tagged record ... Occupant: Employee_Pointer; end record; end Offices; limited with Offices; with Perks; package Employees is type Employee is tagged private; private type Employee is tagged record ... Assigned_Office: Office_Pointer; Bonus_Package: Perks.Goodies; end record; end Employees; with Offices; package Perks is type Goodies is tagged private; private type Goodies is tagged record ... Corner_Office: Office_Pointer; end record; end Perks; Now in my proposal, a cowboy maintainer when he added the Corner_Office to the list of possible perks could go ahead and change the limited with Offices; to with Offices; and assuming that doing so did not cause any naming conflicts, the program would now compile. The careful maintainer would add a subclass, say Executive, to Employee and put it in a separate package that would not need a with or limited with for Offices. Without my rule, the limited with on Employees is now a lie. When another cowboy maintainer comes along and wants to add a with clause to Offices, he will need a fairly sophisticated tool--hopefully a part of the compiler or linker error messages--to tell him what has gone wrong. So as I see it, with my rule (you can't have a limited with Foo; on a unit that already depends on the Foo) USERS should be happy. Without it you may find the same objections as to earlier proposals, that the context clause does not accurately describe the dependencies. It may be that the proposal needs to be further modified to make it easier for Rational to do incremental compilation, but I just don't see it. It is just one more constraint that has to be checked when a unit is modified. ************************************************************** From: Pascal Leroy Sent: Monday, March 31, 2003 4:24 AM > That looks like a second-hand ripple effect to me. It's especially > annoying because P.Index is never used as an incomplete type. I agree that there's a ripple effect here. However it's not too bad from my perspective: if I have to disable incremental compilation in the child units and body when a limited_with is added to a spec, that's a nuisance, but I can live with it: it's not going to have a huge impact. Compare this with having to disable incremental compilation in the entire universe when any with_clause or renaming changes (that would not be acceptable). This being said, it would be better if the language were free of ripple effects. But to me the compelling argument is the breach of privacy: since I don't think that assuming-the-worst is acceptable, I guess I will have to agree (somewhat reluctantly) with Tucker's position. > (The big possible > annoyance with limited with is the lack of control over what becomes > incomplete. Here we see that it can be too much.) Right, but we are not going to give control on a declaration basis. I think the right answer here is that you can use child units to structure your code appropriately: put the declarations that don't need to become incomplete in the parent, and in a child put exclusively those types that need to become incomplete. Of course, you probably don't want to do that on existing code, as it could entail major restructuring. But on new code, this is a perfectly sensible way to organize things. > However, any solution that requires writing two things is going to be > inferior to a solution that requires writing one. So, I suspect that > alternative 6 ("limited with") is likely to carry the day based on > better usability, even if it gives some implementers heartburn. Randy has seen the light. ;-) ************************************************************** From: Tucker Taft Sent: Monday, March 31, 2003 1:29 PM I think we all would say "limited with" has a nice "feel" about it. The big issue is implementability. Eliminating the affects on other compilation units resolves my main concern. I think there might still be some issue for folks with fat and thin pointers (e.g. GNAT), but they don't seem worried, so why should I be? We will have to implement a pretty big chunk of code to build up the limited view from a "quick-and-dirty" parse of a package. I think "type T.C;" could be significantly easier to implement. But if the consensus is to go with "limited with," I am "on board." ************************************************************** From: Robert Dewar Sent: Monday, March 31, 2003 2:24 PM Well GNAT allows the use of thin or fat pointers, although I am not sure this is a problem in any case. ************************************************************** From: Randy Brukardt Sent: Monday, March 31, 2003 5:39 PM > > (The big possible > > annoyance with limited with is the lack of control over what becomes > > incomplete. Here we see that it can be too much.) > Right, but we are not going to give control on a declaration basis. Every proposal other than alternative 6 (limited with) does have control on a declaration basis. Moreover, the lack of such control has been an annoyance in Ada since it was invented (this isn't a new problem with 'limited with'; it's a problem with all with clauses). > I think the right answer here is that you can use child units to structure > your code appropriately: put the declarations that don't need to become > incomplete in the parent, and in a child put exclusively those types > that need to become incomplete. Of course, you probably don't want to > do that on existing code, as it could entail major restructuring. But > on new code, this is a perfectly sensible way to organize things. "Can" is certainly true, but "desirable" is in the eye of the beholder. Here, you're taking the other side (Tuck's side) of the same argument that has typically been used against type C.T;. I don't think proposals should have to rely on "structuring" in order to work; we don't want to impose a structure on users of Ada - they should be allowed to use their own design. In particular, I prefer to keep the ancillary types with the main O-O type. Most O-O types have some types (typically elementary) that are used to help define the operations for the main type - such as discriminants, lengths, etc. I strongly disagree with Robert Eachus' contention that such types indicate that something is wrong with the design. For instance, Claw.Image_List includes: package Claw.Image_List is type Root_Image_List_Type is abstract new Ada.Finalization.Controlled ... type Color_Kind_Type is ( Default_Color_Scheme, Color_4_Bit, Color_8_Bit, Color_16_Bit, Color_24_Bit, Color_32_Bit, Color_Device_Dependent); type Image_List_Index_Type is new Claw.Natural_Int; type Overlay_Index_Type is new Claw.Natural_Int range 0 .. 15; NO_OVERLAY : constant Overlay_Index_Type := 0; function Get_Image_Count (List : in Root_Image_List_Type) return Claw.Image_List.Image_List_Index_Type; -- Returns the number of images in List. procedure Set_Image_Count (List : in Root_Image_List_Type; New_Count : in Claw.Image_List.Image_List_Index_Type); -- Sets the number of images in List. ... end Claw.Image_List; I prefer to keep these extra types with the root type, as they can only be used with that type. Putting them in a separate package just means more times that you have to go searching for the declaration (if you use a lot of use clauses) or more noise in the text (if you use fully expanded names). > > However, any solution that requires writing two things is going to be > > inferior to a solution that requires writing one. So, I suspect that > > alternative 6 ("limited with") is likely to carry the day based on > > better usability, even if it gives some implementers heartburn. > > Randy has seen the light. ;-) I don't think that's quite the right description. It's more that the Ada community as a whole is not likely to give too much weight to the implementability of a feature, and that whatever is nasty about 'limited with' is not going to visible to average users until it was been implemented in compilers for a while. So I thinks users will prefer 'limited with', and there isn't going to be a obvious 'show stopper' for it. On the other hand, unless there is some major revelation when I do the implementation analysis (either way) or a customer with $$$, I don't see RRS being able to afford the implementation costs for 'limited with'. A simplified version of alternative 4 will meet our immediate needs with what appears to be far less work. But, our implementation needs should not stand in the way of what probably will be judged to be the best solution. Tucker said: > I think we all would say "limited with" has a nice "feel" about it. > The big issue is implementability. Eliminating the affects on > other compilation units resolves my main concern. I think there might > still be some issue for folks with fat and thin pointers (e.g. GNAT), but they > don't seem worried, so why should I be? We will have to implement > a pretty big chunk of code to build up the limited view from a > "quick-and-dirty" parse of a package. I think "type T.C;" could > be significantly easier to implement. But if the consensus is > to go with "limited with," I am "on board." Which is pretty much where I stand (with the exception of the remark about "type T.C"). I don't see any reason why 'limited with' couldn't be implemented - the amount of work involved is unlikely to be of interest to users. ************************************************************** From: Randy Brukardt Sent: Monday, March 31, 2003 6:31 PM > Now in my proposal, a cowboy maintainer when he added the Corner_Office > to the list of possible perks could go ahead and change the limited with > Offices; to with Offices; and assuming that doing so did not cause any > naming conflicts, the program would now compile. The careful maintainer > would add a subclass, say Executive, to Employee and put it in a > separate package that would not need a with or limited with for Offices. You didn't show the original program, which is important, because it appears to be illegal. If Perks originally had a 'limited with Offices' in it (or Perks doesn't exist at all), "Assigned_Office: Office_Pointer;" in Employees is illegal, because Office_Pointer is incomplete. In any case, this is not the case that I'm interested in. I was interested in the case where one of the related elementary types that declared with these types gets used. See my answer to Pascal for a real-life example of what I'm talking about. Or, to use your example (made legal assuming AI-230): limited with Employees; package Offices is type Office is tagged private; type Window_Count is range ...; ... -- Operations. private type Office is tagged record ... Occupant : access Employees.Employee'Class; Windows : Window_Count; end record; end Offices; -- Employees remains the same. package Perks is type Goodies is tagged private; ... -- Operations. private type Goodies is tagged record Dental_Insurance : Boolean := False; Disability_Insurance : Boolean := False; end record; end Perks; Now, imagine that we got a call on Friday that we have to add support for a minimum number of windows as a Perk by the end of the quarter (today is a particularly good day for this example :-). The obvious fix which preserves strong typing is: with Offices; package Perks is type Goodies is tagged private; ... -- Operations. private type Goodies is tagged record Dental_Insurance : Boolean := False; Disability_Insurance : Boolean := False; Minimum_Windows : Offices.Window_Count := 0; end record; end Perks; One could argue that the type Window_Count should be moved somewhere else, but that is unlikely for two reasons: 1) The number of windows is a property of an Office - it isn't very meaningful by itself; 2) The time allotted for the change means that major restructuring has to be avoided if at all possible. This is often how I have to do maintenance: when a customer has a critical bug, you have to find a way to fix it, design be damned. If the fix is a real mess, I'll note that and try to schedule some time to redo the code. But that often doesn't happen for months or even years, because that sort of cleanup rarely helps the customers or attracts new ones. > Without my rule, the limited with on Employees is now a lie. ... Not really. Employees is essentially doing a 'with type Employees.Employee;'. The only problem is that 'Window_Type' is getting dragged along as well (a type we don't even use in Employees). That, of course, is a standard problem with with_clauses in Ada, it certainly isn't new here. But we don't want rules that make that clearly obvious. > So as I see it, with my rule (you can't have a limited with Foo; on a > unit that already depends on the Foo) USERS should be happy. Without it > you may find the same objections as to earlier proposals, that the > context clause does not accurately describe the dependencies. Ada with_clauses are too coarse to accurately describe dependencies (unless you restrict yourself to one declaration per package -- which I doubt many people do). I don't see that we can do better (or should try to do better) with 'limited with'. In any case, this maintenance change doesn't suddenly make the original design invalid, and it's unlikely to confuse the programmer. But your rule (or Pascal's, for that matter) might make it a lot harder to fix a problem and preserve strong typing. In the example above, if various units become illegal from adding the 'with Office;', the programmer (who is under severe time constraints) is likely to toss the strong typing and just make the number of windows an Integer. I don't think we want language rules which encourage that. > It may be that the proposal needs to be further modified to make it easier for > Rational to do incremental compilation, but I just don't see it. It is > just one more constraint that has to be checked when a unit is modified. I'd be interested to know Pascal's opinion on this proposal. It seems that he's said that it would be unacceptable, but I'm not certain. Robert's rule would require rechecking all possible dependents [at least to see if they have a limited with] when a with_clause is added, even those that don't directly with the unit. I believe that currently only units that have a with for the changed unit need to be checked, so that potentially would be a significant change. ************************************************************** From: Pascal Leroy Sent: Tuesday, April 1, 2003 3:09 AM This is just another argument against my "revert to incomplete" rule and in favor of Tuck's approach. But then I think I have come to agree that Tuck is right anyway because of the interaction with private types. I actually think that in this example the quick-and-dirty fix is to use Natural for component Minimum_Windows, and to come back later to do more extensive restructuring. The reason is that we are getting pretty close to a situation where a cycle that involved two packages (Employees and Offices) must be extended to involve a third package (Perks). This is particularly true if you take operations into account. For instance, in order to assign offices to employees, the package Offices may have to export an operation like: procedure Check_Consistency (Of_Offices : Office; With_Perks : Perks.Goodies); which would force limited_withs all over the place. This is telling me that this maintenance operation might actually expose shortcomings of the original design, and therefore require extensive rearchitecturing. > Ada with_clauses are too coarse to accurately describe dependencies (unless > you restrict yourself to one declaration per package -- which I doubt many > people do). I don't see that we can do better (or should try to do better) > with 'limited with'. Ada dependencies are at the level of units (well, declarative parts). That has been the case from day 1 and I actually think it would be counter-productive to offer finer granularity for type circularity (among other things Ada is for programming in the large, and I don't see it useful to micro-manage visibility in real-life programs). ... > I'd be interested to know Pascal's opinion on this proposal. It seems that > he's said that it would be unacceptable, but I'm not certain. Robert's rule > would require rechecking all possible dependents [at least to see if they > have a limited with] when a with_clause is added, even those that don't > directly with the unit. I believe that currently only units that have a > with for the changed unit need to be checked, so that potentially would be a > significant change. I didn't reply to Bob E.'s proposal because I don't see how it differs from the classic ripple effect that we have been trying to avoid since we started discussing 217. Adding a with_clause to a unit would require rechecking all the dependents of that unit to see which ones have a limited_with_clause, and in these units locate the constructs that need to become illegal (remember that we manage dependencies at the declaration level, not at the unit level). Not only would this entail a considerable implementation effort, it would essentially negate incremental compilation. The basic tenet of incremental compilation is that the impact of a change is proportional to the size of the change, not to the size of the whole program. We go to considerable lengths to ensure this property. The analysis needed to incrementally detect a dual path would be proportional to the size of the whole program, so it's not acceptable. ************************************************************** From: Robert I. Eachus Sent: Tuesday, April 1, 2003 12:59 PM Ah, but there is a better way to implement my proposal. I think we can agree that the cost of supporting limited withs should be zero if there are no limited withs, and the cost of adding a with clause for a unit that does not have any limited withs should be small. To do this, keep a database of units with limited withs containing the closure of all dependencies for that unit. If a limited with is added to a unit, there is work to be done. If a with is added to a unit that is in the closure database, you can immediately check if it causes an illegality in another unit. If not there is work to be done to extend the closure database with the closures of the current unit. (This can be done at the point the context clause and unit name have been processed.) Also with this approach, units with limited withs but few or no dependencies on non-predefined library units have little or no impact on compilation times of other units. I hope that this will be the common case for units with limited withs. There will be many units with units with limited withs in their closure, but that is not a problem, it is just the closures of units with limited withs themselves that should be kept small. Also note my exclusion of predefined units from the closures. In a heirarchy of units, only levels containing limited withs need this closure database. Oh, as a sort of humorous aside, there isn't much work for a large program with lots of limited withs and few real withs. That is a program structure we might want to visit once we have a set of agreed rules for limited withs--it may make sense. ************************************************************** From: Pascal Leroy Sent: Thursday, April 3, 2003 2:31 AM > If a limited with is added to a unit, there is work to be done. If a > with is added to a unit that is in the closure database, you can > immediately check if it causes an illegality in another unit. There are too many occurrences of the words "with" and "unit" to be sure, but I don't think this works. The pseudo-code for handling the situation where "with P" is added to unit Q should look like: 1: for each unit U that has a limited with loop 2: for each unit V which is mentioned by U in a limited_with loop 3: if U depends (transitively) on Q and 4: P depends (transitively) on V then 5: error; 6: end if; 7: end loop; 8: end loop; First, notice that the outer loop (line #1) has to process all the units that have a limited_with. If you assume that a constant proportion of the units in the program will have limited_with_clauses, this loop runs in O(N). (I'll admit that the proportionality constant may be quite small.) More importantly, to answer the questions at lines #3 and #4 you need to traverse the dependencies of units U and P. The check at line #4 is particularly annoying because (1) it has to be in the innermost loop and (2) if you want to use a database to speed it up, this database has to contain the transitive dependencies of each unit in the program, and that means that the size of the database is in O(N**2). We have customers with tens of thousands of units. An algorithm with a running time in O(N) is unappealing for such large-scale programs. A database of size O(N**2) is just ridiculous. ************************************************************** From: Robert I. Eachus Sent: Thursday, April 3, 2003 6:52 AM I'll use with P; ... package Q is ... to identify the units involved. No when you add the with clause, you probe the database, and the initial probe takes time on the order of log(N). If P is not in the database done. If P is in the database, it may take time proportional to the number of units with limited withs that depend on Q to verify whether or not the unit mentioned in the with is included in these dependences, but I expect that number to be small. Notice that there is a lot of cancelling out that goes on. A unit has to be in the database only if it depends on a unit with a limited with for X, and does not depend on a unit with a direct dependence on X. This is key. In our canonical example, any unit that depends on both Offices and Employees does not need to be in the database. In effect the limited withs cancel out. If P does appear in the database then you need to update the database to add the (limited with) closure of P to the (limited with) closure of Q. This is a join which will take time proportional to the sum of the size of the two closures. In my experience, when the ordinary closures get large, the overall project is in trouble, independent of limited withs and compiler issues. If you think of that added with as a bomb with effect proportional to the size of the closure you can see what happens. When the average number of side effects of a maintenance change is greater than one, any maintenance change has global consequences. But ignoring that for the moment, the worst case for this step is O(N). And if it is O(N), the time has nothing to do with the limited with. You have to compute the closure of Q to insure that it is not circular. The size of the joined closures in the database will be much smaller than the one you have to compute anyway, unless all units have limited withs. > More importantly, to answer the questions at lines #3 and #4 you need to > traverse the dependencies of units U and P. The check at line #4 is > particularly annoying because (1) it has to be in the innermost loop and > (2) if you want to use a database to speed it up, this database has to > contain the transitive dependencies of each unit in the program, and > that means that the size of the database is in O(N**2). > > We have customers with tens of thousands of units. An algorithm with a > running time in O(N) is unappealing for such large-scale programs. A > database of size O(N**2) is just ridiculous. As I said above, well before the database gets that large, the structure of the with clauses will be a maintenance nightmare. But remember that constant of proportionality. Only units that depend on a unit that has a limited with, and do not have a direct dependence on the same unit need to be in the database. If 1 in 100 units has a limited with, the size of the database would be worst case 1/100 O(N**2). But even that is worst case. Units with limited withs may be nearer than other units to the root of the dependency tree, so lets say that it is ten times more likely that a unit will depend on a unit with a limited with. Now the constant of proportionality is 1/1000. And for programs that don't use limited withs, the constant is zero. And that I think has to be considered the final answer. If there are no units in the library with limited withs, the added effort is at worst O(1)--checking to see that there are no limited withs to worry about. If a unit does not depend on a unit with limited withs in a library with limited withs, the time to determine that is O(logN), where N is the number of units with limited withs. As for a program where a large fraction of the units have limited withs, and 100% of the units depend on units with limited withs, that structure is going to be costly. But no more costly than a structure where those withs are replaced by withs. (Of course, the structure with the withs would be highly circular, but that is beside the point. We only have to worry about the compile time effects of such a program if limited withs are such a wonderful feature that "everyone" uses them.) I fail to see that as an argument against the limited with approach. ************************************************************** From: Pascal Leroy Sent: Thursday, April 3, 2003 9:57 AM > Notice that there is a lot of > canceling out that goes on. A unit has to be in the database only if > it depends on a unit with a limited with for X, and does not depend on a > unit with a direct dependence on X. Now you have completely lost me. Aren't you proposing that this case be illegal? In a previous message you wrote 'It should be illegal for a context to include a "limited with P;" clause if the remainder of the context semantically depends on P.' So the case you mention is not canceling, it's just illegal. > But remember that constant of proportionality. > Only units that depend on a unit that has > a limited with, and do not have a direct dependence on the same unit > need to be in the database. If 1 in 100 units has a limited with, the > size of the database would be worst case 1/100 O(N**2). You seem to imply that only the units having limited_with_clauses need to be in the database, and I don't understand this. How do you handle the following: package P is ... end P; with P; package Q is ... end Q; -- with Q; -- Uncomment this. package R is ... end R; limited with P; with R; package S is ... end S; My understanding is that you are proposing that uncommenting the "with Q" on R would make unit S illegal. If that's the case, I don't see how you can detect this case without storing (or recomputing) the dependencies of Q. But then this means that _each_and_every_unit_ needs to be in your database, so the proportionality constant is more like 1. (In essence, I am saying that you need pretty much the whole graph to detect the addition of a second path, and from an information theory standpoint it doesn't sound too surprising.) ************************************************************** From: Robert I. Eachus Sent: Thursday, April 3, 2003 5:03 PM > Now you have completely lost me. Aren't you proposing that this case be > illegal? In a previous message you wrote 'It should be illegal for a > context to include a "limited with P;" clause if the remainder of the > context semantically depends on P.' So the case you mention is not > canceling, it's just illegal. The difference is direct and indirect dependence. All I am proposing is that a unit call it Y that depends on a unit X cannot have a limited with for X. You obviously want the body of a unit whose declaration has a limited for X to be able to have a regular with for X. The point is that the body of Y, if it has a direct with of X is back in the normal Ada universe. Even though it has a dependence on X and a dependence on its declaration which has a limited with for X, you can drop it from the database. > You seem to imply that only the units having limited_with_clauses need to be > in the database, and I don't understand this. How do you handle the > following: > > package P is ... end P; > > with P; > package Q is ... end Q; > > -- with Q; -- Uncomment this. > package R is ... end R; > > limited with P; > with R; > package S is ... end S; > > My understanding is that you are proposing that uncommenting the "with Q" on > R would make unit S illegal. If that's the case, I don't see how you can > detect this case without storing (or recomputing) the dependencies of Q. That is the tricky part. The database needs to contain each unit that depends on a units with a limited with Foo, and as mentioned above, does not also depend directly or indirectly on Foo. In this case, when first compiling everything without the commented with, the compilation of S puts two entries in the database: S --> P, and R --> P. Since R is in the database, recompiling R requires adding its new closure to the database: Q --> P, P --> P (Illegal.) Only units with limited withs, and units that depend on units with limited withs (with the exception explained above) need to be in the database. If a unit declaration has a huge number of with clauses, and/or a large list of direct dependences the project (and your compiler) have major maintenance problems unrelated to limited withs. But for "well structured" Ada programs, bodies often have large closures, declarations (I hope the only place we allow limited withs) don't. In fact, if I were into defining quality measures for Ada code, I would probably use the sum of the size of closures of package and generic specifications divided by the number of such units as one measure. If you want to add in the size of the closures of bodies, there should be a weighting factor that made body closures have one tenth the weight or some such multiplier. > But then this means that _each_and_every_unit_ needs to be in your database, > so the proportionality constant is more like 1. (In essence, I am saying > that you need pretty much the whole graph to detect the addition of a second > path, and from an information theory standpoint it doesn't sound too > surprising.) As I have said, if a large project with hundreds or thousands of units gets to this point, it is already a disaster. I and others on this list can probably tell you horror stories like this. My favorite was one Air Force project where they recompiled the world every night. (First clue of a problem...) They asked the Air Force for advise on a better compiler, because with their current compiler, they were taking over 8 hours to do this and were only 20% of the way into the project. The call got directed to me, and I went over the code structure with the developer. I explained to the developers how to structure an Ada program so that key library packages didn't have to be recompiled so often. (Moving with clauses to the body where possible, replacing constants in package specs with inlined functions and so on.) About a month later I checked back to see how things were going. They were now doing reference builds once a week--good sign--and recompiling the world took less than an hour. But the key news was that error rates were way down and code development was ahead of schedule. ;-) This is why I take 'problems' of slow compilation with a large grain of salt. If the problem can be traced to bad program structure, I'd rather find out about it as soon as possible. ************************************************************** From: Pascal Leroy Sent: Thursday, April 3, 2003 6:20 AM Attached is an attempt at updating AI 217-06 based on the recent discussion of the visibility rules. I am not sure that we really need to outlaw a use_clause for a limited view of a package. It seems to me that the Beaujolais-ish effect that Tuck originally discovered in this area was due to an improper interaction between limited_with_clauses, nonlimited_with_clauses, and visibility. Tuck, what do you think? [Editor's note: This is version /02 of the AI.] ************************************************************** From: Tucker Taft Sent: Thursday, April 3, 2003 10:50 AM I think a nice simplification of the implementation is that it is *only* expanded names that are affected by a "limited with" clause, where the prefix is a package with a limited view. Adding "use" clauses to the mix breaks that, and seems like an unnecessary complexity. Also, I believe we are requiring the limited "with" itself not mention renamings. This means that it is kept as simple and straightforward as possible. I might go further and disallow uses of renamings of limited with packages, rather than just saying they are limited as well. This would really make sure that all uses are clear and unambiguous. But as far as whether the Beaujolais-ish effect is present, I think it is still there if we allow "with P" inside "limited with P.Q;" and we allow "use P" after a limited with of P.Q. ************************************************************** From: Pascal Leroy Sent: Thursday, May 15, 2003 3:29 AM Attached is a document explaining how we would implement "limited with" in our compiler. This was one of my action items from the last meeting (I know, I'm late...). --- Preliminary Remarks Because this document is mostly an internal design document, it refers to details of the implementation of our compiler that are probably incomprehensible to the uninitiated. For the benefit of the ARG readers, I am starting by providing some general information on the terminology used below. Our compilation process is decomposed into three phases: parsing, semantic analysis and coding (coding is not particularly relevant to the implementation of limited with clauses). At the end of the parsing phase the unit is said to be in the _source_ state, and at the end of the semantic analysis it is said to be in the _analyzed_ state. A unit in the source state has a Diana tree with lexical information but no semantic information. A unit in the analyzed state has a Diana tree with all the static semantic information. The _program_library_ is a cache gathering some of the information found in Diana trees. The program library is used to determine which units need to be recompiled without having to open numerous Diana trees. A _decl_#_ is a hash code computed for every declaration in a unit, which describes the static semantics of that declaration. Decl #s serve two purposes: 1 - They are used to build cross-unit reference: a cross-unit reference has the form . 2 - They are used to implement incremental compilation: if a declaration is edited, and its decl # changes, then all the declarations that depend on it must be recompiled. The decl #s existing in a unit are stored in the program library and constitute the _exports_ of the unit. Currently, decl #s are computed at the end of semantic analysis. An _id_table_ is an AVL tree associated with a declarative part that maps an identifier (in the lexical sense) to one or more Diana nodes. Id tables are used for name resolution. Most of the semantic information related to Diana trees is stored permanently. However, there exists information that is never referenced across compilation units, but needs to be recorded on a Diana tree during the course of a single compilation. This information is stored in _temporary_attributes_ of the tree. I think that's all for the terminology. Implementation Model In the course of parsing a library package specification, we construct limited decl #s in the parser reduction actions. Limited decl #s are only built for (visible) types and package declarations. They are only based on (1) the type or package name and (2) the type's tagged-ness. These decl #s constitute the limited exports of the unit, and they are recorded as such in the program library. (Meaning that we probably need to run a minimal exports collection phase after parsing.) When a unit is analyzed, full decl #s are allocated for each declaration (as they are today). These full decl #s don't replace the limited ones, because there may be clients which reference the limited decl #s. Therefore, we may have two decl #s for a given declaration. This is not a problem when following a pointer, but note that when building a reference we must be cautious to use the limited or full decl # depending on whether we have visibility on the limited or full view of a unit. In recompilation terms, consider the situation where a declaration is edited in such a way that its full decl # changes, but not its limited decl #. Only those clients that referenced the full decl # need to be recompiled. This ensures recompilation minimization even in the presence of limited withs. We cannot find ourselves in a situation where the full view of a unit is somehow inconsistent with the limited view of the same unit, because as soon as a declaration is edited in the Ada editor, it is reparsed, its limited decl # is recomputed, and its full decl # is dropped on the floor. The normal incremental compilation mechanisms then kick in to ensure that the proper clients get obsolesced. I don't foresee any issue with timestamps and the like: after parsing (and limited decl numbering) has taken place, a unit looks pretty much as if it had been compiled and then edited, so again incremental compilation should save the day. Also as part of parsing, id tables must be built for each declarative part that is part of the limited view. These limited id tables only contain the names of types and packages. They are used for name resolution in clients which only see the limited view of the unit. Note that they are not overwritten when the unit is analyzed, because it is possible to reference an analyzed unit through a limited with. So for some Diana nodes we'll have two id tables. At the beginning of the semantic analysis of a unit, the context clauses of that unit and of its ancestors are processed. If a limited with clause is encountered before a nonlimited with clause, we records this fact, so that name resolution is later performed using the limited id table (we might need a temporary attribute to facilitate this, like we do for private with). When the name of a type is resolved through a limited id table, a temporary attribute is set on that type to record the fact that its type spec is effectively incomplete, even though it might look like a full type spec in the Diana tree. This attribute must be consulted when checking for premature usages of incomplete types. (We already have such an attribute for checking premature usages of private types, so I expect that this part will more-or-less fall out naturally.) Note that supposedly we do not have to bother with this attribute for types declared in other packages (e.g., an access-to-incomplete type declared in an intermediate package). Say that such a type was compiled against the limited view of a unit. If we see the full view of that unit, following Diana pointers through the intermediate unit will leads us to the full type definition, which is what we need. If we only see the limited view of the original unit, then we should never need to follow any Diana pointer through the intermediate unit, because the corresponding source constructs ought to be illegal. Of course, I fully expect that there will be cases where we do too much look-through, but these should be fixed, as they probably correspond to know-too-much bugs. There are some contexts (e.g., use clauses) where the name of a unit referenced through a limited with is illegal. Presumably this is a legality rule, not a name resolution rule, so it is checked pretty much like the rules related to private with clauses, i.e., at the point where we decorate a name, after name resolution has been performed. The compilation ordering algorithm must be modified to take limited with relationships into account. Before analyzing a unit, its full closure (i.e., the closure obtained by following both limited with clauses and nonlimited with clauses) must be in the source state. Its normal closure (i.e., the one obtained by following only the nonlimited with clauses) must be in the analyzed state. There is no prerequisite for promoting a unit to the source state (this is good, it means that you don't do compilation ordering when hitting the Syntax button). When a unit is demoted to source, its full decl #s, full id tables and full export lists are deleted, but the limited decl #s, limited id tables and limited export lists must be retained. The coding phase should be unaffected by all this. Before a unit is coded, the compilation ordering algorithm must ensure that its full closure is analyzed. This guarantees that coding never lands in an unsemanticized unit. ************************************************************** From: Tucker Taft Sent: Thursday, May 15, 2003 8:37 AM Thanks for doing this. I guess I must be late too (I forgot about this part of the assignment). Perhaps Randy can do his usual excellent job of reminding me of the remaining items on my todo list for the upcoming ARG meeting ;-). As you may have noticed, the AdaUK folks showed a surprisingly strong preference for the "type C.T" approach, followed by the "limited with." Since I presented the material, part of this might be that my bias showed through, though I tried to be impartial (other observers there, such as John Barnes, might be able to comment on my success), and frankly, I like the "limited with" proposal from a user point of view. Some of the reasons cited for preferring the type C.T approach were that it seemed to be a smaller change that was easier to understand, and that it didn't introduce any new kinds of module interdependence. It also seemed to be more "structured" to some members of the audience. For what that's worth... ************************************************************** From: Randy Brukardt Sent: Tuesday, June 17, 2003 2:12 AM Here is the implementation report I promised at the last meeting for all three live AI-217 proposals. I know that these are really late, but at least you'll have three days in which to read them. (Of course, if you're at Ada Europe already, that probably is really about 3 minutes. :-( [Editor's note: The report can be found in the !appendix to AI-217-07.] ************************************************************** From: Ed Schonberg Sent: Thursday, July 10, 2003 12:14 PM At the last ARG meeting, the following example was discussed: package P is type T is record ... end P; ----------------------------------------------------------------- limited with P; | with P; package Q is | package R is type Acc_Lim is access P.T;| procedure Proc (X : P.T); X : Acc_Lim; | end R; end Q; | ----------------------------------------------------------------- with Q, R; limited with P; -- (1) procedure Main is begin R.Proc (Q.X.all); -- Should be legal end; ----------------------------------------------------------------- It was agreed that the call was legal, because the context of the call provided the full view of type T, and the explicit dereference, although it yielded the limited view, was allowed to "be aware of" the full view. Furthermore, the presence or absence of the limited with_clause (1) did not affect the legality of the call. On the other hand, there was some agreement that, in the same context, with the Limited_With present, R.Proc (Q.T'(Q.X.all)); was illegal, because within S, Q.T can only denote the limited view, and an incomplete type cannot be the prefix of a qualified expression. THE rule would seem to be that a direct use of the type name only sees the limited view, but uses of the type that come from the semantic closure are legal if the non-limited view is available (although the rules will probably not mention the term semantic closure). Consider now the following variation on the previous example: package P is type T is record ... end P; ----------------------------------------------------------------------------- limited with P; | limited with P; | with P; package Q is | package R is | package S is type Acc_Lim is access P.T;| procedure Proc (X : P.T);| ... X : Acc_Lim; | end R; | end S; end Q; | | ----------------------------------------------------------------------------- with Q, R; with S; -- (2) limited with P; procedure Main is begin R.Proc (Q.X.all); -- legal? end; Without the with_clause (2), this should be illegal, because the full view of the type is nowhere available (both Q and R have limited_withs on P). However, once the with_clause on S is present, the semantic closure does make the full view of P available. Following the previous example, this would be legal. Either this is counterintuitive (and unpermissibly alcoholic) or else the previous attempt at stating the rule is incorrect. Guidance is welcome. The query comes in connection with a prototype implemen- tation of Limited_With that Javier Miranda and I have been constructing within GNAT. The implementation so far has been straightforward, and provides the desired functionality for all straightforward examples tested so far. We hope that the precise semantics of pathological cases is less of a thicket than it appears to us right now! ************************************************************** From: Gary Dismukes Sent: Thursday, July 10, 2003 1:21 PM > Furthermore, the presence or absence of the limited with_clause (1) did > not affect the legality of the call. Right, the call is legal because the expected type of the parameter is the full view (that's the "context of the call"). > On the other hand, there was some agreement that, in the same context, > with the Limited_With present, > > R.Proc (Q.T'(Q.X.all)); > > was illegal, because within S, Q.T can only denote the limited view, and > an incomplete type cannot be the prefix of a qualified expression. My understanding as well (though you mean P.T, not Q.T in the above). > THE rule would seem to be that a direct use of the type name only sees the > limited view, but uses of the type that come from the semantic closure are > legal if the non-limited view is available (although the rules will probably > not mention the term semantic closure). I believe that the model we discussed was based on whether the full view is available via an expected type, not based on whether the full view is in the semantic closure. If it were based on the semantic closure then ripple effects could occur, which we're trying to avoid. > Consider now the following variation on the previous example: > ... > with Q, R; > with S; -- (2) > limited with P; > procedure Main is > begin > R.Proc (Q.X.all); -- legal? > end; > > Without the with_clause (2), this should be illegal, because the full view > of the type is nowhere available (both Q and R have limited_withs on P). Right. > However, once the with_clause on S is present, the semantic closure does > make the full view of P available. Following the previous example, this > would be legal. Either this is counterintuitive (and unpermissibly alcoholic) > or else the previous attempt at stating the rule is incorrect. No, the semantic closure shouldn't be used here, only the expected type should be taken into account. At least that was my understanding of the model discussed at the meeting. ************************************************************** From: Tucker Taft Sent: Thursday, July 10, 2003 1:21 PM > THE rule would seem to be that a direct use of the type name only sees the > limited view, but uses of the type that come from the semantic closure are > legal if the non-limited view is available (although the rules will probably > not mention the term semantic closure). I think the rule is rather that a dereference that delivers an incomplete type is allowed so long as it is used in a context where the corresponding full type is a) immediately in scope, or b) declared within a (non-limited) with'ed package, or c) provided by the current complete context of overloading (e.g. as the expected type in this example). Mere presence in the "semantic closure" is *never* sufficient. > Consider now the following variation on the previous example: ....[See it above - ED] The declaration of procedure Proc would only be legal if T were a *tagged* type, since only tagged incomplete types may be used as formal parameter types. Presuming the example is changed to have T tagged, then... > with Q, R; > with S; -- (2) > limited with P; > procedure Main is > begin > R.Proc (Q.X.all); -- legal? > end; The legality is unaffected by the presence of the "with" of S, since it doesn't directly contain the declaration of the full type (the semantic closure is irrelevant). Since none of the 3 possibilities I gave above are satisfied, this should be considered illegal. However, I forget whether we allowed for a special case for a dereference producing a *tagged* incomplete type. We didn't want to allow that for dynamically tagged controlling parameters, because those require fetching the tag for dispatching or a tag check. I'm not sure what we decided about the case of non-controlling or statically-tagged parameters. If we allow it, then that becomes a case (d) to be added to the cases (a)..(c) given above. ... The key message is presence in the semantic closure is never sufficient. All rules must be based on what is directly with'ed, or in the immediate scope, or in the overloading context. ************************************************************** From: Randy Brukardt Sent: Thursday, July 10, 2003 3:16 PM > Since none of the 3 possibilities I gave above are satisfied, this should > be considered illegal. However, I forget whether we allowed for > a special case for a dereference producing a *tagged* incomplete type. > We didn't want to allow that for dynamically tagged controlling parameters, > because those require fetching the tag for dispatching or a tag check. > I'm not sure what we decided about the case of non-controlling or > statically-tagged parameters. If we allow it, then that becomes > a case (d) to be added to the cases (a)..(c) given above. It could be valuable to allow this for class-wide parameters (where there is no real dereference, since tagged types are pass-by-reference). The same is true of statically bound calls. The important thing is to avoid any dereference where there is access to the object itself (as in accessing the tag or discriminants), because we don't know enough about the components to do anything. It's hard to say how that rule should be written, though. ************************************************************** From: Ed Schonberg Sent: Thursday, July 10, 2003 2:56 PM The first two cases are clear. The third one (the overloading context) presents serious implementation problems, at least for us. Our implementation assumes that at any given time in the analysis there is only one view of the type that is available. That is either the limited view Tl or the non-limited view Tn. When we obtain the expected type from the context, for example the formal of a subprogram, there is no indication of the visibility that the type may have had at the time the subprogram declaration was analyzed. It's just T, whose attributes are either those of Tl or Tn. If the current view is Tl, it is possible to indicate that Tn is available (if it showed up in the analysis of the closure) but it is never the case that two constituents of an expression have different views of the same type. To go back to the first example: package P is type T is tagged record... end P; limited with P; -- (1) with P; -- (2) package Q is package R is type Acc_Lim is access P.T; procedure Proc (X : P.T); X : Acc_Lim; end R; end Q; with Q, R; limited with P; procedure Main is begin R.Proc (Q.X.all); end Main; By the stated rule this is legal, but it would be illegal if (1) and (2) were exchanged (because the context provided by the call would then be the limited view, right?). However, when we analyze Main, these two cases are indistinguishable (in our implementation of course, but I thought that Janus and Rational had a similar view of things). Therefore, the proposed rule does not seem to be implementable for us, without major changes in data structures (or a sudden blinding flash of inspiration!). ************************************************************** From: Tucker Taft Sent: Thursday, July 10, 2003 3:30 PM Perhaps you can treat the incomplete types made available in a limited view of a package as truly distinct types that just happen to match a particular full type during overload resolution. Clearly you must have rules that allow two different types to match under certain circumstances. For example, T'Class and T are clearly different but they match in certain contexts. Similarly, certain named access types can be used in contexts where the expected type is an anonymous access type. Incomplete types can only be used in a few places (e.g. as designated types, and if tagged, as formal parameters). These places seem well enough defined that having a special matching rule wouldn't be too bad, presuming you can manage to treat them as truly distinct types in most cases. ************************************************************** From: Robert I. Eachus Sent: Friday, July 11, 2003 11:46 AM I hate it when this happens--but it is just the way my mind works. I think/hope that the original case has been addressed. But there is another case that looks troublesome: package P is type T is tagged record ... procedure Proc (X: T); end P; -------------------------------------------------------------------------------------------- limited with P; | with P; | with P,R; package Q is | package R is | package body Q is type Acc_Lim is access P.T'Class; | type Child is new P.T; | begin X : Acc_Lim; | procedure Proc (X : Child); | X := new Child; end Q; | end R; | end Q; -------------------------------------------------------------------------------------------- with P, Q; limited with R; procedure Main is begin Proc (Q.X.all); -- which Proc is called? end; I think that the "right" answer has to be that R.Proc gets called via dispatching. There is a possibility if I had put the call elsewhere that Program_Error could occur, but that is nothing new. I don't think there are any cases where type T could be private and only visible through a limited with, and the access type is visible through a with. *************************************************************** From: Tucker Taft Sent: Friday, July 11, 2003 11:46 AM I don't know what the issue is here. package body Q is illegal because it isn't needed by its spec, but presumably you add a pragma Elaborate_Body or some sort of dummy procedure which will cause it to be allowed, it seems clear that by the time you get to Main, Q.X points at an object with tag R.Child (you need to add a "use" for R in the body of Q as well). In general, whether you use limited with or with has no run-time effect, except perhaps elaboration order, so what exactly is your question? *************************************************************** From: Robert I. Eachus Sent: Saturday, July 12, 2003 6:32 PM >I don't know what the issue is here. package body Q is illegal because >it isn't needed by its spec, but presumably you add a pragma Elaborate_Body >or some sort of dummy procedure which will cause it to be allowed, Oops, consider it done. >it seems clear that by the time you get to Main, Q.X points at an object >with tag R.Child (you need to add a "use" for R in the body of Q as well). Just to make sure we are on the same wavelength, assume I change the body of Q to: with P,R; package body Q is begin X := new R.Child; end Q; to fix that. >In general, whether you use limited with or with has no run-time effect, >except perhaps elaboration order, so what exactly is your question? Sorry if that wasn't clear. It is possible for a limited with to result in a (run-time) elaboration order that results in additional ABE checks being necessary. I think I have convinced myself that these are already language defined checks, but that get optimized away currently. *************************************************************** From: Ed Schonberg Sent: Sunday, September 28, 2003 4:04 PM !standard 10.01.02 (03) 03-04-03 AI95-00217-06/02 !standard 10.01.02 (04) !standard 10.01.02 (08) !standard J.10 (00) !class amendment 03-02-04 !status work item 03-02-04 !status received 03-02-04 !priority Medium !difficulty Hard !subject Limited With Clauses This is a summary of our implementation of Limited_With clauses in GNAT. In a nutshell: after a few false starts, the implementation proved to be straightforward, thanks to a critical suggestion from Tucker. The changes to various phases of the compiler were very localized, and amounted to no more than a few hundred lines of code. We reused some of the machinery of our implementation of now-discarded With_Type clauses. The current prototype implementation is complete, except for one exception mentioned below. Overview -------- GNAT is a source-based compiler. If unit U1 has a limited_with clause that mentions unit U2, then U2 must be present in the environment. U2 is parsed, and its abstract syntax tree is constructed. The tree is not analyzed, but undergoes a simple processing that identifies type declarations in the visible part of the unit (and the visible part of nested packages). These type declarations are used to created shadow entities in the tree, that receive the semantic annotations that describe incomplete types. These shadow entities are made visible while the limited_with_clause is effective. In a context where a regular with_clause on U2 is effective, the shadow entities are invisible. Most of the implementation code handles the installation and removal of the shadow entities and their counterparts, within the data structures that implement the visibility rules. There are small changes to the basic type-checking routine, to some binder routines, and to the parser. These changes amount to less than 50 lines of code. Semantics. ---------- The semantic complications introduced by this construct arise from cases of multiple access paths: in a given context, an entity may be available through a limited_with_clause, as well as through a regular_with clause, a renaming, direct visibility in a child, etc. The proposed legality rules for limited_with_clauses make some of these cases illegal, and otherwise try to minimize the disruption that the addition of such a clause might have on an existing program. At first sight, the limited_view of a type is analogous to the partial view of a private type, and a similar implementation model suggests itself. In GNAT, for entities with multiple views, it is always the case that there is a single tree node (a defining_occurrence) that holds the current view of the entity. In the case of a private type, the occurrence in the private declaration (that is to say, the specific node in the abstract syntax tree that is the defining_identifier of the declaration) is used to denote the entity. Its contents are swapped with the semantic information of the full view when needed (i.e. when compiling the private part, the package body, or the private part or body of a child unit), but at all times it is the same node in the syntax tree that embodies the identity of the type. Our first implementation attempt therefore treated the limited_view in the same way: the semantic information attached to the entity was either incomplete or complete, but the same node of the AST was used in all cases. However, this model cannot handle correctly cases of multiple access paths, because it assumes that at a given point, only one of the possible views is available. However, there are cases when two components of a construct denote different views of the same type. The most common usage of limited_with_clauses involves multiple access paths: if a specification for package P has a limited_with_clause in order to import some type T from package Q, it is usually the case that the body of P will have subprogram bodies that manipulate T, and thus the body of P will have a regular with_clause on Q. Thus, in the body of P, the type T is available both through the limited_with_clause on the library_unit, and the regular with_clause on the body. In such a case the desired semantics are obvious: the non-limited view of T is available. In other cases, the semantics are more subtle, as in the following example: Consider a package A that declares a type TA, a package P that has a limited view of A, and a package Q that has a non-limited view of A. Both P and Q declare acces types whose designated type is TA. In context where both P and Q are visible (but A might not be), objects designated with these access types must be compatible, even though one of them denotes the limited view of TA and the other the non-limited view. As a consequence, we must treat the limited_view of a type as an entity in its own right, and add special type resolution rules to make the limited and non-limited views compatible. We introduce shadow entities to hold the limited_views, and links between the shadow entity and its partner, to simplify type-checking. It turns out that these shadow entities (which collectively correspond to the "abstract" of the package) make the implementation simpler that the one-entity model with a swapping mechanism between views. They also make it easier to maintain necessary invariants between the semantic analyzer and the back-end (that is to say the tree transducer that uses the Ada AST to build the GCC tree). Implementation details. ----------------------- The changes to compiler internals are only of interest to GNAT maintainers. It may be worth listing the following: a) There were small changes in the contents of AST nodes for with_clauses, for package declarations, and for type declarations. b) The package that handles compilation units and the processing of context clauses was obviously the most affected. The following subprograms were added: Build_Limited_View: to create an incomplete (shadow) entity. Install_Limited_Withed_Unit: to make the incomplete entities visible Remove_Limited_With_Clause: to remove the incomplete entities from visibility Expand_Limited_With_Clause: to generate limited_with clauses on the ancestors of a unit that is mentioned explicitly in a limited_with clause (as long as those ancestors are not mentioned in an effective regular with_clause). In addition, the following existing routines had to be modified to make use of the above (names should be self-explanatory): Analyze_Compilation_Unit, Analyze_With_Clause, Install_Context_Clauses, and Remove_Context_Clauses. c) The package that handles type declarations was modified to retrieve the non-limited view (when available) of the designated type of an access type that denotes the limited view. d) There were a few modifications to the binder, to prevent limited_with clauses from creating semantic dependencies (that's the whole point, after all!). e) Finally, there were small modifications to error checking and reporting: half-a-dozen error messages were added, and some error conditions disabled when dealing with limited views. To test the implementation on the examples proposed in various versions of this AI, we also modified the processing of subprogram declarations, to allow tagged incomplete formal parameters. All the examples compiled properly (or were rejected correctly). Pending. -------- The single gap in the implementation is the rule that requires the closure of a partition to include the body of a unit that is mentioned in a limited_with clause, when there is no regular with_clause on that unit elsewhere in the partition. This is a bind-time rule that is simple enough to implement: we did not have time to include yet, and suspect that partitions that require it will be execeedingly rare. *************************************************************** From: Tucker Taft Sent: Sunday, September 28, 2003 4:26 PM Cool! Glad I made a useful suggestion. Which one was it? *************************************************************** From: Ed Schonberg Sent: Sunday, September 28, 2003 8:17 PM That the limited views be full-blown new entities, rather than transient attributes of existing ones. As usual, there is no design problem that can't be solved with an additional level of indirection. In any case, thanks again for sending us on the right path! ***************************************************************