!standard 3.09.01 (03) 04-02-29 AI95-00344/01 !class amendment 03-08-07 !status work item 04-02-29 !status received 03-04-21 !priority Medium !difficulty Hard !subject Allow nested type extensions !summary Type extensions are allowed in scopes more nested than their parent type. Checks are performed on function return and on allocators to ensure that objects of such a type extension do not "escape" to a less nested environment. !problem The restriction against nested type extensions makes it harder to instantiate certain generics in a local scope, if those generics internally use extension from some kind of standard library-level type, such as a storage pool, or a controlled type, or a stream. Such restrictions can "snowball," meaning that many generics in a system may only permit library-level instantiations. !proposal Repeal the accessibility check in 3.9.1(3). Replace it by something that still allows generic sharing (probably prohibiting extension in generic bodies). Add accessibility checks on class-wide allocators and return statements. !wording Delete the last two sentences of 3.9.1(3). Add after 4.8(5): If the designated type of the type of the allocator is tagged, the accessibility level of the designated type shall not be statically deeper than that of the type of the allocator. Add after 6.5(20): If the result type is class-wide, a check is made that the accessibility level of the type identified by the tag of the result is not deeper than that of the master that elaborated the function body. If this check fails, Program_Error is raised. !example !discussion [Editor's note: 3.9.1(4) is retained (which prevents extension in generic bodies), because the contract model issues discussed in the AARM remain. You can get similar issues from parameters which depend on formal types. See the e-mail below.] IMPLEMENTATION ISSUES When a type is extended in a more nested scope, the dispatching operations may need access to data declared in this nested scope. This will generally require a static link or global display. However, the caller of such a dispatching operation is generally "unaware" that they are calling such a nested dispatching operation. This means that the dispatching operation must deal with the static link or global display internally, using information that the caller passes in. The simplest approach seems to be as follows: When defining the record layout for a nested type extension, an additional implicit component is allocated to contain the static link, or a pointer to a saved global display, or in the case of a shared generic, a pointer to an instantiation descriptor in which the static link or global display can be found. !corrigendum 03.09.01(03) !ACATS test !appendix From: Tucker Taft Sent: Monday, April 21, 2003 3:44 PM The current rule that disallows extending a type at a more nested scope than the parent type has always been a bit troublesome. I believe that during the Ada 9X process we felt quite strongly that this was the only safe rule. However, as I have thought about this rule more recently, it seems to be possible to avoid problems by simply performing accessibility checks at allocators and function returns. The danger from nested extensions is that a global variable will end up with a tag of a nested type extension. But this can only happen in two ways: 1) A function with a classwide result type returns an object of a type extension declared within it. 2) An allocator for an access-to-classwide type declared at an outer level is evaluated for a more nested type extension. Both of these situations seem eminently checkable. (1) is very similar to the current accessibility check on return-by-reference objects. (2) is similar to a check that is required when the designated type for an allocator has an access discriminant initialized by local_obj'access. Is there some other problem that I have forgotten? Should we consider an amendment to allow nested extensions? It would seem to eliminate a number of other annoying problems where certain generics cannot be instantiated at a nested level. **************************************************************** From: Randy Brukardt Sent: Monday, April 21, 2003 4:15 PM > Should we consider an amendment to allow nested extensions? No. > It would seem to eliminate a number of other annoying problems > where certain generics cannot be instantiated at a nested level. No, this is the same as saying that we're eliminating all useful generic sharing. (I define "useful" generic sharing as that done without looking into the body. If you can look into the body, there is no need or reason for language rules to be involved, the sharing 'contract' now is essentially the entire program.) It is critical that access-to-subprogram types (which include tags) never, ever 'leak' out of the generic body (because such subprograms have a 'different' profile than the 'apparent' profile used for checking language rules). The accessibility rules happen to have that effect; but clearly they were not designed for that. If the accessibility rules are weakened or removed, you simply have to add them back in some other way. Or, simply drop generic sharing and the contract model. (I see no reason to keep any of the generic-sharing rules if any of them are eliminated.) I started to write up a long LSN/web page on the topic of generic sharing, because I seem to need to re-explain it every couple of months. Other things intervened, but it appears that I still need to do so... **************************************************************** From: Tucker Taft Sent: Monday, April 21, 2003 5:15 PM Would it help if you still had to put type extensions in the spec as opposed to the body (if they are at the same accessibility level as the spec, that is)? I thought that was the thing which simplified generic sharing. Allowing extensions inside a subprogram body seems like a different sort of thing, and I presumed would not break sharing. **************************************************************** From: Randy Brukardt Sent: Monday, April 21, 2003 5:37 PM The only way that it wouldn't break sharing is if there never, ever was a way to call them from outside of the body (including from calls outside of the body). That clearly includes dispatching calls, but is not limited to that. I don't see what such a type (or object) would be useful for. Indeed, I can't see what your proposed extension is useful for. Your proposed limitations seem to prevent the type from being used for dispatching. That presumably includes implicit dispatching (that is, finalization and storage pools), since that is likely to be implemented with real dispatching (they certainly are in Janus/Ada). If it's not prohibited from use in dispatching, I don't see how you expect it to work - you'd have to include a static link or display in the tag - indeed in ALL tags. (The 'current' one wouldn't work, you could make external calls to the same level and have the wrong link/display values. That's not a problem for local variables because they're not accessible, but certainly it would be possible with a dispatching call where the lack of visibility doesn't matter). That's clearly a lot of overhead (because you'd have to 'switch' the display or static link on every dispatching call); there had better be a pretty solid purpose behind the extension. **************************************************************** From: Tucker Taft Sent: Monday, April 21, 2003 6:45 PM Yes, the static link/display is the issue, I suppose. It would seem you could implement this by putting the link/display information in any object of a nested type extension. We do something analagous for dealing with nested protected types, I believe, because we pass a pointer to the protected object in the same register we normally use for passing the static link. If the protected type is itself nested, the "real" static link is retrieved from the protected object. Given that nesting is usually shallow, even a display would only occupy a few words in the tagged object. Does that sound feasible? **************************************************************** From: Randy Brukardt Sent: Monday, April 21, 2003 5:37 PM Well, that would work, but its completely non-responsive to my question. In Janus/Ada, all objects have a static size. Anything dynamically sized is allocated from a storage pool and stored in a fixed size descriptor (which is often just a pointer). Thus, there is only the options of including enough space for everything (8 levels in Janus/Ada) in all objects or allocating the space off of one of the storage pools (with the associated runtime overhead). But this is equivalent to putting it into the tag, just a lot more expensive in space overhead (unless there is only one object). So I don't see any difference. Moreover, it does nothing to address the distributed runtime overhead: every dispatching call has to extract the tag or display from somewhere (doesn't matter where), and arrange to use it on the call. That's not too expensive for a static link, but it is very expensive for a display (because you have to save the old one somewhere, then copy in the new one, then reverse the process on return). And there is no hope to optimize it out (unless you're into link-time code generation), because any program can have a nested type added in the future. The absolute best you can do is check at runtime whether you have to do it, but even then, there is a gob of code needed to provide the rarely-to-be-used support (and the extra test and jump). That's the definition of distributed overhead. > Does that sound feasible? Feasible, but not practical as noted above. Do you have any realistic example of what this would be good for? I still see no value to it, and you haven't explained how you would use it. You have to explain what is so important to take on the sort of distributed overhead that you are discussing here. I'm not going to waste any more time discussing implementation until I understand what it would be used for. Otherwise, I'm just answering the same old questions; I have no opportunity to suggest an alternative that might not be such a problem. **************************************************************** From: Tucker Taft Sent: Monday, April 21, 2003 8:38 PM I'm not sure I understand why this would introduce a dynamic size. If inside a subprogram body I extend a type declared outside the subprogram, I know I am creating a nested extension, and only objects of that type need a static link/display, and I "know" exactly how deeply nested I am. (I wouldn't want to put them in the type descriptor pointed to by the tag because we allocate and initialize that statically.) When dispatching operations of this type are called, they are treated as though they were declared at the same level as the root type as far as the caller is concerned. If they need access to data at levels more nested than that, they have to go indirect via the static link/display stored in the tagged object. I don't think this implies distributed overhead, since only dispatching operations for nested extensions would have to use the "funny" static link/display entries. > ... > > Do you have any realistic example of what this would be good for? I still > see no value to it, and you haven't explained how you would use it. You have > to explain what is so important to take on the sort of distributed overhead > that you are discussing here. I'm not going to waste any more time > discussing implementation until I understand what it would be used for. I agree that if it implies all of the distributed overhead you suggest, then it would not be a good idea. In fact, if it involved any distributed overhead at all it would be a bad idea. But it seems like the overhead could be kept within the place where a nested extension appears. Disallowing such things in generic bodies would seem a reasonably limitation, since type extensions are not permitted in generic bodies now anyway. But I presume the benefits are obvious. Right now, if you define a root type like "Finalization.Controlled" or something else, and then create useful code that deals with the root-type'Class, this is only usable with types declared at the same nesting level as the root type. **************************************************************** From: Randy Brukardt Sent: Monday, April 21, 2003 9:31 PM > I'm not sure I understand why this would introduce a dynamic size. If > inside a subprogram body I extend a type declared outside > the subprogram, I know I am creating a nested extension, > and only objects of that type need a static link/display, > and I "know" exactly how deeply nested I am. > (I wouldn't want to put them in the type descriptor pointed > to by the tag because we allocate and initialize that > statically.) OK, but how do you know whether objects of that type have such a thing, and where it is?? This can't happen by magic: a dispatching call needs exactly the same layout of components that it is going to deal with in all cases. > When dispatching operations of this type are called, they are treated > as though they were declared at the same level as the root type > as far as the caller is concerned. If they need access to data > at levels more nested than that, they have to go indirect via > the static link/display stored in the tagged object. You want a wrapper around the wrapper? Yikes - I can't figure out how this works as it is. That is going to be one messy wrapper, because it has to have a stack frame of its own (to save the current display in); which means the parameters will have to be copied. > I don't think this implies distributed overhead, since only > dispatching operations for nested extensions would have to > use the "funny" static link/display entries. No, I suppose you're right (presuming that such wrappers can be built - I think there might be problems if the root type is nested. What is the level of the wrapper in that case? I suppose one can just forget such a case, 'cause it's useless.) > ... > Disallowing such > things in generic bodies would seem a reasonably limitation, > since type extensions are not permitted in generic bodies now anyway. But that's a fallacy. The only reason type extensions are prohibited in generic bodies is because of the assume-the-worst accessibility check. You're proposing to eliminate the accessibility check; then there is no longer any legitimate reason for prohibiting type extensions in a generic body. We'd have to add a specific rule for that - and it would have no real justification - it would essentially be there only because of me. People would picket my office if they found out. :-) > But I presume the benefits are obvious. Right now, if > you define a root type like "Finalization.Controlled" or > something else, and then create useful code that deals > with the root-type'Class, this is only usable with types > declared at the same nesting level as the root type. Maybe to you, but I don't see the "benefits" of declaring a nested type. The only times I've wanted to do it is in Q&D test programs where declaring a separate package is a pain. But that 'pain' is a benefit in real, large systems - it prevents the main subprogram from containing stuff that really doesn't belong there. Moreover, if you declare a nested type, it can't (usefully) be inherited from -- which means you're losing some of the benefits of O-O. So, you almost always want a package for your type extensions. If the goal is to get subprogram-level finalization, I'd rather see that done in a first-class manner (which would be both easier to write and more obvious that it is unusually expensive). That requires some syntax. But I see no benefit at all to extending anything other than Finalization.Limited_Controlled in a subprogram, and that being the case, I'd rather leave well enough alone and only deal with the case that is interesting. **************************************************************** From: Pascal Leroy Sent: Wednesday, April 23, 2003 5:07 AM I don't think you have yet provided a compelling justification for this change. Other than the specific case of Controlled (which we might want to address separately) I don't see that nested OO hierarchies are particularly useful or frequent in practice. I would not be opposed to reopening AC-00050, but this is going to irritate Randy. **************************************************************** From: Tucker Taft Sent: Thursday, April 24, 2003 10:15 AM > ... But I > see no benefit at all to extending anything other than > Finalization.Limited_Controlled in a subprogram,... I'm surprised others haven't run into this before. One of the biggest "privacy" breakages is that a generic can't be instantiated inside a subprogram if it happens to implement any of its private types using type extension of some type declared outside the generic. This is particularly annoying for "container"-type generics, where one would really like to be able to instantiate and declare a local hash table, for example, but can't do it if it happens to declare a local storage pool type. It is often possible with some amount of pain to move the instantiation, while keeping the local hash table object (presuming the instantiation declares a type rather than havings its own internal table object). But this gets nastier if you are yourself writing a generic, and the element type is a formal type. Ultimately you end up with a whole bunch of generics that can only be instantiated at library level. **************************************************************** From: Randy Brukardt Sent: Monday, March 1, 2004 9:12 PM Tucker wrote in the AI: [Editor's note: Version /01.] > When defining the record layout for a nested type extension, an > additional implicit component is allocated to contain the static link, > or a pointer to a saved global display, or in the case of a shared > generic, a pointer to an instantiation descriptor in which the > static link or global display can be found. Of course, this doesn't work in the generic body for all of the reasons that previously been discussed. The accessibility check in generic bodies prevents all kinds of things that can't be implemented -- and the problems arise from issues that have nothing to do with accessibility. Every subprogram that can be dispatched to or accessed via an access-to-subprogram needs a thunk: both to add the hidden generic parameter (which we've talked about at length), and also to convert the parameter passing between the implementation for the formal and the implementation for the actual. If you were to put the instantiation descriptor into tagged objects that are extended in generic units, you could declare a wrapper that added that to the call for dispatching calls. And the tagged object is always passed by reference, so that's OK. But other parameters might have different passing, and you couldn't reconcile that in a wrapper (you don't know the actual types involved). Which brings up an interesting question: is a operation (potentially) primitive if it depends on a formal type? We never had to answer this in Ada 95 for bodies; we said that it is true in specs. Consider something like: type Root is tagged ... procedure Oper (Obj : in out Root; Arg : in Natural); generic type Priv is private; package Gen is ... end Gen; package body Gen is type New_Type is new Root with ...; procedure Oper (Obj : in out New_Type; Arg : in Priv); --?? ... end Gen; package Inst is new Gen (Natural); Now, does Inst.Oper override the inherited one? If this occurred in the spec, it would. But if you do this in the body (which is currently illegal), then sharing without body knowledge is impossible, because you need to know the actual type(s) in order to determine the contents of the tag. (One also has to wonder about the contract model implications here.) **************************************************************** From: Tucker Taft Sent: Monday, March 1, 2004 9:42 PM > Of course, this doesn't work in the generic body for all of the reasons that > previously been discussed. Sorry, I really don't understand your model enough to suggest something intelligent. In any case, I was presuming that the restriction 3.9.1(4) would remain: A type extension shall not be declared in a generic body if the parent type is declared outside that body. Perhaps that restriction makes no sense if we are allowing type extension in arbitrary non-generic bodies, but I presumed it was still important for shared generics for some reason. ... > package body Gen is > type New_Type is new Root with ...; > procedure Oper (Obj : in out New_Type; Arg : in Priv); --?? This would violate 3.9.1(4) I believe. > ... > end Gen; **************************************************************** From: Ramdy Brukardt Sent: Monday, March 1, 2004 9:58 PM > A type extension shall not be declared in a generic body > if the parent type is declared outside that body. Sigh. I didn't remember this rule; I've always thought that the accessibilty check itself preventing body tagged types. Anyway, the AARM primarily discusses accessibility reasons for this rule. Those obviously would be gone. But it also discusses problems with abstract subprograms causing contract problems. Those would still remain, so we'd still have a reason for the rule (although we'd need to rewrite the explanation in order for it to make sense). I think you could construct an example where whether the program was legal or not depended on the actual type used for another parameter similar to the example I showed in my last message. So it looks like there are real contract problems with that, so there isn't a problem. But I do think a brief discussion of why 3.9.1(4) remains (and that it exists!) would be valuable in the AI. Otherwise, people might be expecting more out of this than they'll actually get. **************************************************************** From: Tucker Taft Sent: Thursday, April 1, 2004 11:19 AM Randy, Steve Baird, and I have been having a bit of a side-bar discussion on paragraph 3.9.1(4) and how it relates to AI-344 on allowing extending in nested scopes. The most recent version of AI-344 did not touch 3.9.1(4) which disallows extending in a generic body a type defined outside it, since we would be allowing extension in all kinds of other contexts. After a few back-and-forths with Randy, we came up with the following wording for 3.9.1(4) which seems to be satisfactory, and avoids contract model issues associated with extending types with unknown number of functions that require overriding (due to a controlling result). It also allows slot numbers to be assigned for any new operations, since the parent is not a formal type. Within the body of a generic unit, or the body of any of its descendant library units, a tagged type shall not be declared as a descendant of a formal type of the generic unit. This is carefully worded to handle child units, as well as interfaces (hopefully). The net effect is that you *can* extend a type declared outside the generic body, so long as it is not a descendant of a formal. The anticipated implementation model is still that the new type would have a component that pointed to a block of information which would contain static link/ display/instance descriptor/whatever. Wrappers for the dispatching operations would load this information into appropriate registers, etc., and then call/jump to the user-defined primitive subprograms. These are essentially the same thing as "trampolines" which are used by GNU C and friends to support nested functions. I have attached below the mail sent by Steve Baird, Randy, and myself, for the record. If there are no major objections, I will include this change in the next version of AI-344. ---------- From Steve, 3/29/04, 6:54 PM: I strongly want to be able to instantiate the various container generics anywhere (e.g. inside of a subprogram which is inside of a generic body), even if they are implemented using controlled types. This runs afoul of 3.9.1(4) . If you've already addressed this problem, then this message can be ignored. Given AI-344's runtime checks, the only remaining need for 3.9.1(4) stems, as I understand it, from the case described in AARM 3.9.1(4.a-j). This case can only arise if the parent type is abstract. Could 3.9.1(4) be relaxed so that it only applies to abstract types? I'm not claiming this is elegant, but it would allow an important case. ---------- From Tuck to Randy and then his reply (3/31/04): >> Hi Randy, >> There seems to be some interest in pushing forward AI-344, nested type >> extensions. Paragraph 3.9.1(4) disallowing extending in generic >> bodies has been an issue. It seems somewhat arbitrary, if we allow >> extending pretty much anywhere else. I was thinking that the following >> might solve the problem: >> >> Replace 3.9.1(4) with: >> >> A type extension shall not be declared in a generic body if the parent >> type is a formal type of the generic. >> >> Would this be implementable in your compiler? Notice that I allow the >> parent to be descended from a formal type, but not a direct use of the >> parent type. That avoids the danger that the parent type is class-wide. >> >> Presumably with AI-251, this would become: >> >> A type extension shall not be declared in a generic body if the parent >> type or one of the ancestor interface types is a formal type of the >> generic. >> >> Can you think about this a bit and let me know? I thought about this some last week while I was working on the minutes. One comment is that I never saw any actual reaction to keeping 3.9.1(4). The only person who seemed to care was you. So I'd suggest not giving this rule an importance out of proportion to reality. (After all, most types can be hoisted to the spec.) First of all, it will be very hard to implement any part of AI-344 in our compiler. We implicitly add generic descriptor parameters for all subprograms (declarations and calls) if they are declared in a generic. Doing this causes the other parameters to be renumbered automatically. The implementation you've suggested for AI-344 requires this not to be true for wrappers for tagged types. That will require major restructuring in the compiler - either the wrappers would have to be created by a totally separate set of code (meaning the duplication of the 1500 lines of call generation code - sounds very bad) or the automatic, under the covers operation of generics would have to be gotten rid of. Neither is likely to be practical; I'd expect that a likely implementation of AI-344 would completely disallow it in generics. In any case, I don't want to stand in the way of progress here (unless it is clearly going too far, like the generic instantiation junk). So most of the discussion is more hypothetical than what actually can be implemented. It's about what could be implemented with heroic efforts to provide a usable solution, as opposed to what's clearly impossible or outrageous. Besides implementation problems, that rule is needed to avoid contract model issues. I hadn't thought about class-wide; the AARM mainly discusses it in terms of must-be-overridden functions. The example is something like: generic type Foo is tagged private; package Gen is ... end Gen; package body Gen is type New_Foo is new Foo with private; -- ?? end Gen; type Bar is tagged ...; function A_Bar return Bar; package New_Gen is new Gen(Bar); --?? Since A_Bar is a primitive function returning an object of Bar, it must be overridden. But how would New_Foo know to do that? It's clear that the type cannot depend on a formal. There is also a practical contract violation here. We have a meta-rule that tags always can be created statically. We certainly adhere to that rule by generating the tags for types declared in a generic at instantiation time. There is no way to do that for a type that depends on a formal that declared in the body. (And note that here, this is true even if there is an intermediate type declared in the specification, because that cannot tell us all of the overriding needed.) generic type Foo is tagged private; package Gen is type Foolike is new Foo with null record; end Gen; package body Gen is type New_Foo is new Foolike with private; procedure Some_Operation (A : New_Foo); --?? end Gen; type Bar is tagged ...; function Some_Operation (A : Bar); package New_Gen is new Gen(Bar); Some_Operation for New_Foo is overriding and must be replaced in the tag for New_Foo. But how is an implementation supposed to figure that out? It cannot know whether Foo has a primitive operation Some_Operation. This could be "solved" by declaring Some_Operation to be not overriding. But that would violate the principle that a generic works the same as a non-generic. (We certainly aren't going to declare this as non-overriding outside of a generic, that would be a gratuitous incompatibility.) Moreover, there isn't any way I can think of to create this tag at runtime (short of including the entire Ada symboltable and code in the program, which isn't remotely practical). Because you'd have to pass the name and parameter and result profile with every slot in order to figure out any if there is any overriding. (Usually we assign slot numbers to these, but in this case that's not possible.) This gets even more interesting when there is a package spec in the body: package body Gen is package Nested is type New_Foo is new Foolike with private; procedure Some_Operation (A : New_Foo); --?? end Nested; end Gen; Here, Some_Operation is clearly a primitive of New_Foo. How do we assign a slot number? It might override something that comes from the formal, or it might not. So I'm certain that any relaxation of the rule would have to disallow anything that depended on a formal in any way. (Direct dependence is not enough.) I haven't been able to decide if derivations from unrelated types (such as "Controlled") are a problem or not. Clearly, you could use the same techniques that you use for non-generic types. The problem (if there is one) would occur if you need any local data space for the wrappers. This I don't really know. At the meeting, we discussed the fact that the wrappers probably would need a runtime data structure (linked through the dynamic stack, perhaps, or perhaps on the finalization chain) to find the appropriate display or static link. Presuming that implementers sign up for that overhead (which is an open question to me, it seems significant, but at least not distributed), then that mechanism would provide a mechanism for finding any data needed. So on that basis, it seems like it *might* be possible to implement. (Of course, generating the wrappers would be a big problem because of the generic parameter issue mentioned previously. But the question is that is would be possible.) So I'd probably not oppose strenuously a rule like: A type extension shall not be declared in a generic body if the parent type is descended from a formal type of a generic. (Note that we have to worry about nested and child cases here, too. It doesn't matter whose formal it is, we don't want to be dependent on it. But we don't want this to apply to types declared in instances, so I'm not sure if the above wording is right. Perhaps we need to say something about "the generic or its ancestors".) P.S. I presume that we'll attach this discussion to the AI? I don't want to have to write this note again!!! ------------------- From Tuck to Randy and then his reply again (3/31/04): ... >> Since we disallow use throughout the generic body, I think this handles >> nested generics and subunits (since they are "plugged" into their stub >> prior to legality checks). Children seem like a problem as you say, >> since their bodies are not considered nested within the ancestor. >> The "ancestor" of a generic is not always well defined, so >> perhaps the following: >> >> Within the body of a generic unit, or the body of any of its descendant >> library units, a tagged type shall not be declared as a descendant >> of a formal type of the generic unit. >> >> This hopefully covers interface types as well. I think this works. **************************************************************** From: Randy Brukardt Sent: Friday, April 2, 2004 9:24 PM Unfortunately, I've thought of another case where there is trouble creating a tag in the body without knowing the actual types involved. Consider the following: type Base is tagged ... procedure Proc (Obj : in out Base; Arg : in Integer); -- (1) subtype Short_String is String (1..10); generic type Frob is private; package Gen2 is ... end Gen2; package body Gen2 is type Der is new Base with ...; procedure Proc (Obj : in out Der; Arg : in Frob); --(2) ... end Gen2; package Short is new Gen2 (Short_String); package Int is new Gen2 (Integer); In Short (1) and (2) have different profiles, and thus overload each other. Therefore (2) gets a new slot (which can be calculated at compile-time, given the above rule). In Int, however, (2) overrides (1). That means (2) is supposed to get the slot of (1). In simple cases like this one, it probably would be possible to figure out the slots by some sort of run-time tag modification. If there were multiple such parameters, though, it could get pretty messy. There also is a practical problem in our implementation. The parameter passing implementation of Arg in (1) would be copy-in; while for Arg in (2) it would be by-reference with the copy made for by-copy types at the call site. Thus, if the wrapper for (2) was just stuck into the slot for (1), it would have the wrong parameter passing code. Not good. I'd suggest some sort of rule to fix this problem, but I can't think of anything off-hand that doesn't throw the baby out with the bathwater. The easy rule (the tagged type cannot have any primitives which depend on generic formals) would prevent pretty much everything useful. And for types like Controlled, it doesn't matter anyway, since the base type doesn't have any operations that could cause problems. But a rule that only eliminated problem cases would have to talk about the intersection of possible profiles of all of the primitives, which seems awful to describe and implement. Something like "Within the body...., a type extension is illegal if a primitive operation of the type has one or more operands which depend on a formal type of a generic, and the primitive could override another primitive of the type in any possible instantiation of the generic." (along with the previous restriction, of course). I think this would still allow containers and Controlled in the bodies of generics (presuming the containers were derived directly from Controlled), but would avoid the problem cases. Alternatively, we could simply say that such "overrides" don't override in a generic body; but of course, that would mean that making a unit generic could change its semantics. I don't like that much (and I'd think that template implementations would like it less). **************************************************************** From: Robert I. Eachus Sent: Friday, April 2, 2004 10:08 PM I guess I see the problem, but I don't see it. The problem occurs only if there is a call inside the generic: Proc(Some_Der, N); -- where N is of type Integer. In the first instance Short, this call goes to the (derived) Proc declared inside Gen2. Your assumption in the Int case is that this call should go to the explicitly declared Proc. But I don't see how it can. The only visible Proc inside the generic with an Integer parameter is the implicitly declared one. If Der and Proc were declared in the generic package spec, then there could be external calls where the overriding must be recognized. But that happens only if the overriding is in the spec, which is not the case in question here. Now if Der was declared in the spec, and there was an explicit Proc in the body, that could cause a problem. However, I think a rule that said that such an overriding was not visible outside the (generic) unit would follow the principle of least surprise. I think this is the weird case Randy is worried about. However, I think that it may be one where a more general fix is needed. Notice that a "normal" overriding in the body of a generic is fine. It is a visible overriding inside the generic as well as externally. The compiler knows that it is an overriding and can deal with it accordingly. In this "accidental" overriding case, only if the author of both the generic body and of the instantiation is the same (Ada language lawyer) programmer, would an overriding visible (only) outside the generic be anything other than surprising. **************************************************************** From: Randy Brukardt Sent: Friday, April 2, 2004 11:27 PM No, the problem also occurs when there is a dispatching call on Base'Class. Which Proc gets calls for an object of Der? That call doesn't have to be inside of the generic; any class-wide call will exhibit the difference in behavior. Note that the use of overridding indicators would show the problem, as neither "overriding" nor "not overriding" would be legal on Proc (assume-the-worst in a generic body). But they wouldn't help... In any case, I'm mostly concerned about the implementation effects on an implementation that tries to share bodies. If there is "conditional" overriding in the body, sharing is going to be very hard or impossible. And since none of this is visible in the spec (we're only discussing 3.9.1(4), which only applies to types completely declared in the body), there can be no body-agnostic sharing in such a case. **************************************************************** From: Tucker Taft Sent: Saturday, April 3, 2004 7:41 AM > Unfortunately, I've thought of another case where there is trouble creating > a tag in the body without knowing the actual types involved. ... I think we would want to treat this analogously to the situation where a type has a private primitive. It's slot is not reused on a type extension even if a primitive with an equivalent profile is declared outside the scope of the private primitive. Instead, the new primitive gets a new slot. In this case, I would say that in a generic body, if a parameter's types might be different in some instances, then the primitive gets a new slot in all instances. I can't imagine any value to having overriding vs. overloading depend on the actuals, since no code outside the instance is going to be able to name this type. Yes it could see such an object at run-time via 'Class, but I don't think that argues for making the overriding vs. overloading instance-specific. In the example you give below, the declaration of "Proc" (2) in the generic body would not override the inherited primitive "Proc" (1), and in fact wouldn't even be a primitive. If Der were declared in a nested package spec, then the new Proc (2) would be a new primitive of Der and get a new slot, even if Frob turned out to be Integer. > ... Consider the > following: > > type Base is tagged ... > procedure Proc (Obj : in out Base; Arg : in Integer); -- (1) > > subtype Short_String is String (1..10); > > generic > type Frob is private; > package Gen2 is > ... > end Gen2; > > package body Gen2 is > type Der is new Base with ...; > procedure Proc (Obj : in out Der; Arg : in Frob); --(2) > ... > end Gen2; > > package Short is new Gen2 (Short_String); > package Int is new Gen2 (Integer); > > In Short (1) and (2) have different profiles, and thus overload each other. > Therefore (2) gets a new slot (which can be calculated at compile-time, > given the above rule). > > In Int, however, (2) overrides (1). That means (2) is supposed to get the > slot of (1). Naaah. Overriding should be something that is decided before instantiation for generic bodies. > In simple cases like this one, it probably would be possible to figure out > the slots by some sort of run-time tag modification. If there were multiple > such parameters, though, it could get pretty messy. Blech. I can't imagine any value to allowing overriding vs. overloading to depend on the actual when in a generic body. > ... > Alternatively, we could simply say that such "overrides" don't override in a > generic body; but of course, that would mean that making a unit generic > could change its semantics. You aren't just making the unit generic (i.e. wrapping the word "generic" around it). You are also taking one of the types and making it a formal type. To me that implies you want the behavior to be "generic" across changes in such a type. You don't want it to suddenly have a differnt behavior if the type happens to match some parameter type of some primitive of some tagged type that happens to be used inside the body. In fact, if you did that, you would create an interesting capability to do a run-time test for (untagged) type identity, which is *not* something we have ever provided in a generic, and I don't think we want to start now! > ... I don't like that much (and I'd think that > template implementations would like it less). Having a template implementation doesn't change my view. Overriding vs. overloading within the body should be decided when you compile the generic, not at the time of the instance. **************************************************************** From: Randy Brukardt Sent: Saturday, April 3, 2004 4:46 PM Tucker said, replying to me: > > ... > > Alternatively, we could simply say that such "overrides" don't override in a > > generic body; but of course, that would mean that making a unit generic > > could change its semantics. > > You aren't just making the unit generic (i.e. wrapping the > word "generic" around it). You are also taking one of the > types and making it a formal type. To me that implies you > want the behavior to be "generic" across changes in such > a type. You don't want it to suddenly have a differnt > behavior if the type happens to match some parameter type > of some primitive of some tagged type that happens to be > used inside the body. In fact, if you did that, you would > create an interesting capability to do a run-time test for > (untagged) type identity, which is *not* something we have > ever provided in a generic, and I don't think we want to > start now! OK, that seems like a good argument. > > ... I don't like that much (and I'd think that > > template implementations would like it less). > > Having a template implementation doesn't change > my view. Overriding vs. overloading within the body > should be decided when you compile the generic, not > at the time of the instance. Of course, this means further futzing with the overriding rules. That doesn't seem too pleasant, either. But the goal (being able to use controlled types and containers in generic bodies) seems worthwhile. **************************************************************** From: Tucker Taft Sent: Sunday, April 4, 2004 1:26 PM I'm not sure that's true (though you may have something specific in mind). The rules I can find generally say that everything relating to overloading and overriding is decided when you compile the generic body. Visible part of instances are different, but the body seems pretty uniformly to rely on overloading/overriding decisions made when the generic body is compiled. E.g. 8.3(13), 8.3(26/1), 12.3(14), 12.3(18). **************************************************************** From: Pascal Leroy Sent: Monday, April 5, 2004 6:36 AM But this is not new futzing. As Tuck pointed out, declarations in the body should work like declarations in the private part, and you probably do quite a bit of futzing already to implement 12.3(18). So you have a firm foundation on which to build even more futzing. **************************************************************** From: Randy Brukardt Sent: Tuesday, April 6, 2004 4:22 PM No, I was thinking about in the Standard, not in our compiler. But you and Tucker have pointed out 12.3(18), so the Standard is already correct on this point. No futzing needed. Some implementers might need to futz if they don't follow 12.3(18) properly, but that's not a problem for this group to worry about. **************************************************************** From: Robert I. Eachus Sent: Monday, April 5, 2004 1:46 AM >No, the problem also occurs when there is a dispatching call on Base'Class. >Which Proc gets calls for an object of Der? That call doesn't have to be >inside of the generic; any class-wide call will exhibit the difference in >behavior. My head is starting to hurt. Right now, Der is declared nested in a generic package body. You are proposing to assign it to a "global" object of type Base'Class, so that it can be operated on outside the generic. But there is no way to do that directly, you have to have an access type declared access (all) Base'Class. Now you can use an allocator for this access type inside the generic, and assign its value to an object of the access type. See RM 3.10.2(14&15). So it seems to me that it would be much easier to do any required magic as a part of that allocator. An allocator for a class-wide type seems pretty magic to me anyway. In fact, in exactly the case we are discussing, the magic rules in 3.10.2(20) and 3.9.1(3) seem to combine to allow, for example, type extension in a generic body, the generic body can be instantiated inside some procedure, and assignment of a value created by allocator to an access object in the procedure body would be legal. In that case, there could be for example a generic formal subprogram, that gets inherited in the generic, and must be dispatched to. So I guess to me it seems silly to distribute overhead to any case other than allocators in package bodies for class-wide access types which allocate an object of a tagged type declared in the generic package body. Furthermore since the painful cases require that the allocator be in the sequence of statements for such a generic package, why not just outlaw it and be done. (If you have an object of the class-wide access type in the package body, with an initial value, that object can only be seen within the package body. It is assignments in the sequence of statements at the end of the generic package body that can cause problems, and it seems to me to be simple enough to treat that scope as statically deeper than the rest of the generic package. I have created packages with a sequence of statements at the end, I have even done it with generic packages. But assignments outside the generic--of an object of a type created in the generic BODY? Come on... >Note that the use of overridding indicators would show the problem, as >neither "overriding" nor "not overriding" would be legal on Proc >(assume-the-worst in a generic body). But they wouldn't help... > >In any case, I'm mostly concerned about the implementation effects on an >implementation that tries to share bodies. If there is "conditional" >overriding in the body, sharing is going to be very hard or impossible. And >since none of this is visible in the spec (we're only discussing 3.9.1(4), >which only applies to types completely declared in the body), there can be >no body-agnostic sharing in such a case. > I don't know what the "best" way to deal with this is. But I think Tucker and I are leaning the same way--allow the useful functionality even if we have to keep some restrictions. Allowing derived types in generic package bodies while not allowing assignment of access values to objects of the derived type outside the generic seems like a very livable restriction. **************************************************************** From: Randy Brukardt Sent: Monday, April 5, 2004 1:45 PM > My head is starting to hurt. Right now, Der is declared nested in a > generic package body. You are proposing to assign it to a "global" > object of type Base'Class, so that it can be operated on outside the > generic. Not at all. First of all, 'Der' isn't neccessarily nested; it's in the body, but could be at library level depending on the instantiation. But in any case, 'leaking' the object out isn't necessary. All you need to do is call a class-wide routine on the base class with a parameter of type Der. If that routine then does a dispatching call on Proc, you can tell whether it's been overridden or not. In any case, Tucker's argument seems strong enough to avoid the issue in the first place, by simply saying that such things are not overriding. *That* we can implement without trouble in a shared implementation. ****************************************************************