Version 1.3 of ais/ai-00363.txt

From: Tucker Taft
Sent: Saturday, November 15, 2003  5:30 PM

Along with Pascal, I was assigned homework to study the problem of access
subtypes.  Here are my thoughts:

First, let's identify the basic problem and sequence of
piecemeal solutions:

-------------------------------------------------------
---- the problem and current solution attempts --------
-------------------------------------------------------

Normally when a discriminated type has defaults, objects of
the unconstrained subtype allow their discriminants to
change as part of a whole-object assignment.

Unfortunately, the *possibility* of constrained access subtypes
means that objects that might be pointed-to by a value of
such a subtype must be constrained, since the constraint check
associated with such a subtype is performed only on assignment
and parameter passing, not on use.

When we added general access types and objects explicitly declared "aliased"
in Ada 95, we permitted access subtypes on these types as well, and
generalized what was originally a requirement only on objects created by
allocators, to apply to all aliased objects, whether declared or
the result of an allocator, as well as all aliased components.

This has created a stream of headaches, particularly related to
aliased components, because composite assignments and view
conversions generally do no per-component constraint checks, so
extra care needs to be used to prevent the possibility of
discriminants changing due to operations on the enclosing
object.  This has led to 3.6(11) which requires aliased components
of non-limited types to be constrained.  However, if the type
is private and its discriminants are only visible in the private part,
then 3.6(11) provides no protection, so 3.7.1(7) was created to
disallow general access subtypes of types whose partial view
lacks discriminants, while the full view has defaulted discriminants.

Furthermore, view conversions of limited array types could
create a problem, if the target and source type differed
in whether the components were aliased, since in one view
the discriminants were mutable, and in the other they weren't.
This led to AI-168 and changes to 4.6(12).

Generic formal (limited) array types create similar problems,
if the actual and formal differ on component aliasing, for essentially
the same reason.  12.5.3(8) worries about the case in one direction,
and AI-275 was created to worry about the case in the other direction.

It was also noticed that 3.6(11) does not provide protection for
generic formal private types, since the actual might have
defaulted discriminants, so AI-295 was created to deal with this
situation, essentially requiring 3.6(11) to be enforced in the
private part of the instance, and enforced in the body of the
instance by a (pseudo) run-time check.

Another somewhat unrelated problem, which was exacerbated by
an Ada 95 change, is that if a private type is implemented
with a full type with defaulted discriminants, then the
implementation might work great in tests that don't put
such objects in the heap, but would suddenly start failing
if a user of the private type decided to allocate objects
of the type in the heap.  This was exacerbated in Ada 95
because we made it easier for a private type with no
visible discriminant part to be implemented with a
discriminant-with-defaults full type.  This was possible
in Ada 83, but required jumping through a few extra hoops.

I know about this problem because a few years ago it hit
some of our customers, because we had implemented Unbounded_Strings
with a variant record, one variant for "short" strings, and
one variant for longer strings.  It is easy enough to solve
by wrapping the full type in a record, but it seems a clear
violation of the "spirit" of privateness, namely that objects
of a private type with no visible discriminants should suddenly
behave differently when clients of the type happen to allocate
them in the heap.

It seems clear that fixing this problem would require allowing
unconstrained heap objects to be created, at least when using
an access type declared outside the scope of the full type,
and disallowing access subtypes of such an access type, even
if the type is pool-specific.  Of course these access subtypes
would be a bit weird, since they would have to be declared
somewhere within the scope of the full type to be able to
see the discriminants, even though the main access type were
declared outside the scope of the full type.

------------------------------------------
---- So where do we go from here? --------
------------------------------------------

The "fix" in 3.7.1(7) which disallows general access subtypes
in certain cases is a bit frustrating, in that it is really just
an attempt to patch a hole in 3.6(11), and doesn't really help
with the problem I described above about private types that stop
working for heap objects.  Unless you know that there are *no*
access subtypes, whether pool-specific or general, can you
start expecting an allocator for a private type with no visible
discriminants to create an object that "works" the same way
as a declared object or component having the same type.
So personally, I would put some priority on finding a
solution that fixes this privacy-breaking situation.
(I'll come back to this later -- see below.)

In addition, the language rules are excessively complicated due to the
possibility of general access subtypes designating types, private or
not, that have defaulted discriminants.  The rules would be significantly
simplified if declared objects and components behave the same whether
aliased or not, as far as constrained-ness.  For these cases,
we are only worried about general access subtypes, since only
general access types can designate (aliased) declared objects or components.

In the discussion of AI-295, one of the alternatives considered was
disallowing creating access subtypes of general access types, perhaps
only if the designated type has defaulted discriminants.  However, this was
felt to be a contract model problem, since a formal *pool-specific*
access type can be associated with an actual *general* access type.
A related problem is that a formal type without defaults can
be associated with an actual type that has defaults.
Let's look at this problem in more detail:

The contemplated rule (in place of the existing rule in 3.7.1(7)) is:
  A discriminant constraint is not permitted on a general access type
  if the designated type has defaulted discriminants.

To enforce this rule in the presence of generics, we would clearly
need to check it in instances, including in the private part.
In the body, we would have to disallow a constraint on any access type
that *might* have this property, or do it as a run-time check.
None of these rules seem too bad, since the likelihood of declaring
access subtypes inside a generic body are quite slim, and if
subtypes of some access type are really going to be used widely, it may be
reasonable to omit the defaults on the discriminants to begin with.

One other alternative to consider is to allow such access subtypes
(perhaps only in generic bodies), but define their semantics to
be that the checks are, as now, made on assignment and parameter passing,
but that, in addition, discriminant checks are also required on use.  On use,
the implementation would be permitted to check for an exact
match to the associated discriminant constraint, or to perform
just the "normal" discriminant checks associated with referencing
discriminant-dependent components.

In fact, probably no wording changes related to discriminant checks
would necessarily be required to adopt this "alternative" semantic model
(other than perhaps to permit the "fiercer" check), since discriminant
checks are defined to occur on every use already, and it is just
logical inference which allows compilers to currently omit the checks
when dereferencing a value of a constrained access subtype.
This logical inference would no longer be true in all cases, if
we permit aliased declared objects and components to have mutable
discriminants.

Clearly disallowing the access subtypes themselves is preferable
from the point of view of doing as much checking as possible at
compile-time, but the "altered" semantic model would actually be a
bit more upward compatible, since existing exception-free code
would remain legal and exception-free.  Code that is currently
raising exceptions might raise them at different places, namely
on use of an access value, rather than on an attempt to change
the discriminants via assignment.

My recommendation?

Disallow the discriminant constraint on a general
access type if the discriminants have defaults on the
designated type, recheck in an instance and presume the worst
in a generic body.  I think this is quite reasonable since
declaring access subtypes is rare to begin with, even rarer when
the designated type's discriminants have defaults, rarer still
in a package body, and probably exceedingly rare in a generic package body.
(If by chance some upward incompatilibility does arise, two "workarounds"
are to eliminate the defaults on the discriminants (if not really needed),
or eliminate the constraints on the declaration of the access subtypes.
Exception-free code should continue to run exception-free.)

This change would allow us to eliminate 3.6(11), and the various
attempts to "patch" it via AI-168 for 4.6(12), AI-275 for 12.5.3(8),
and AI-295 for generics.  Aliased components would no longer
have to be constrained; we could independently decide whether
standalone aliased objects declared with defaulted discriminants would
be constrained by their initial value, though there doesn't seem much
remaining justification for making them so.

-----------------------------------------------------
---- fixing privacy-breaking constrainedness --------
-----------------------------------------------------

As mentioned above, an existing error-prone situation
is that a private type that works perfectly well for
non-aliased objects, starts failing on objects declared
aliased or allocated in the heap.  The "aliased" part could
be fixed by the above change, but the heap-allocated problem is
not fixed by disallowing constraints on certain general access subtypes.

To fix this problem, we must disallow constraints on *all* access types
declared outside the package defining the private type (presuming the
private view has no visible discriminants), allocating
space for unconstrained objects in the heap even when
constraints are given in the allocator for such an access type,
and setting the 'Constrained attribute False when dereferencing
values of such an access type.  Essentially it makes an allocator
for such an access type equivalent to declaring a variable
of the (private) type, which is always going to have mutable
(non-visible) discriminants.

In fact, I believe that constraints are *already* disallowed on
such an access type, even when inside the scope of the full
type of the designated type, because according to 3.10(14), such
an access type is constrained from the get-go.  This is because
at the point of its definition, and throughout its immediate scope,
its designated type never has any discriminants, so there is
nothing to constrain.  As usual, even if some client
knows more than what is known where the access type is declared,
it can't take advantage of that.

So the real change is to make the semantics of allocators (see 4.8(6)) match
this model as well.  Allocators for such an access type should
create unconstrained objects, even if the place where the allocator
takes place might "know" that the designated type has discriminants,
and might use a constrained subtype in the allocator.

If the access type is a general access type, and we presume the "other"
change proposed above is adopted, then this "new" semantics for allocators
should probably apply to access types declared inside the package
as well, presuming the designated subtype is not constrained,
since subtypes would be disallowed on the "inside" general access types
as well.  Furthermore, there would need to be rules disallowing
a 'Access that delivers a value of such a type being applied to a
constrained variable (3.7.2(27)), and disallowing conversion from some other
access type that had constrained designated objects (4.6(16)).
Both 3.7.2(27) and 4.6(16) would then say "discriminated and indefinite"
rather than "discriminated and unconstrained."

I think this further change would be a "good" thing, as it provides
significant improvements to the privacy model, and it complements
the rule suggested which applies only to general access types.

****************************************************************

From: Pascal Leroy
Sent: Monday, November 24, 2003  9:17 AM

Thank you for writing this up, and sorry for the late answer.

I very much like your proposal, in particular the notion of fixing the
privacy-breaking oddity where objects of a private type work differently
depending on whether they are allocated on the heap on declared as normal
variables/constants.

I think that change #1 (discriminant constraints for general access types) is
fairly uncontroversial as this area of the language is quite muddled, so it's
only going to break code which was only working by a fluke.  And the effect of
the change will be to make existing code illegal.

On the other hand change #2 (the privacy-breaking problem) is more dicey as it
introduces a performance incompatibility: imagine a unit that was allocating
1,000,000 objects of 10 bytes, but would occasionally allocate an object of
1,000,000 bytes.  Suddenly it's going to allocate 1,000,000 objects of
1,000,000 bytes.

I am still willing to swallow this incompatibility given the headaches that
it's going to solve, but it's definitely going to be a contentious proposal.

I will include a summary of your proposal in my next WG9 report.  In all
likelihood it will go back to the ARG for further analysis.

****************************************************************

From: Tucker Taft
Sent: Monday, November 24, 2003  1:41 PM

> ...
>
> I think that change #1 (discriminant constraints for general access types) is
> fairly uncontroversial as this area of the language is quite muddled, so it's
> only going to break code which was only working by a fluke.  And the effect
> of the change will be to make existing code illegal.

Once I sat down and began to piece together the set of patches we made to
try to fix the problems with 3.6(11), I was somewhat appalled.  I hope
we can at least get consensus on this part of the proposal.

>
> On the other hand change #2 (the privacy-breaking problem) is more dicey as
> it introduces a performance incompatibility: imagine a unit that was
> allocating 1,000,000 objects of 10 bytes, but would occasionally allocate an
> object of 1,000,000 bytes.  Suddenly it's going to allocate 1,000,000 objects
> of 1,000,000 bytes.

Yes, that is an issue.

I see this problem as relatively rare, since from the outside view, this type
only comes in one size, and all local or global variables are going to be the
"max" size (for most run-time models).  There are also a couple of potential
workarounds:

  1) Declare the (full and private) type limited -- I would presume only
     nonlimited types would have unconstrained instances in the heap.
  2) Put discriminants, known or unknown, on the private type declaration.

> I am still willing to swallow this incompatibility given the headaches that
> it's going to solve, but it's definitely going to be a contentious proposal.

Probably...

****************************************************************