Version 1.4 of ai12s/ai12-0009-1.txt

Unformatted version of ai12s/ai12-0009-1.txt version 1.4
Other versions for file ai12s/ai12-0009-1.txt

!standard A.16(3/2)          15-10-11 AI12-0009-1/03
!standard A.16(36.1/3)
!standard A.16(98/2)
!standard A.16(112.1/3)
!standard A.16(125/3)
!standard A.17(3/2)
!standard A.17(8/3)
!standard A.17(23/3)
!standard A.17(25/2)
!standard A.17(28/2)
!class Amendment 11-11-08
!status work item 11-11-08
!status received 11-10-01
!priority Low
!difficulty Medium
!subject Iterators for Directories and Environment_Variables
!summary
Support for generalized loop iteration for Ada.Directories and Ada.Environment_Variables is added.
!problem
Ada.Directories and Ada.Environment_Variables contain iterator procedures with call-back procedures. There should be a version of the Ada 2012 iterators for these subprograms, so that users can write a loop.
!proposal
(See wording.)
!wording
Modify A.16(3/2) "with Ada.IO_Exceptions;
with Ada.Calendar;
{with Ada.Iterator_Interfaces;}
package Ada.Directories is"
Append after A.16(36/3) the following
-- Directory Iteration
type Directory_Listing is tagged limited private
with Constant_Indexing => Current_Entry,
Default_Iterator => Iterate, Iterator_Element => Directory_Entry_Type;
function Entries
(Directory : in String;
Pattern : in String; Filter : in Filter_Type := (others => True)) return Directory_Listing;
type Entry_Presence is new Boolean;
function Has_Element (EP : in Entry_Presence) return Boolean is (Boolean (EP));
package Directory_Iterators is new Ada.Iterator_Interfaces (Entry_Presence, Has_Element);
function Iterate (Listing : in Directory_Listing) return Directory_Iterators.Forward_Iterator'Class;
function Current_Entry (Entries : in Directory_Listing; EP : in Entry_Presence) return Directory_Entry_Type with Pre => Has_Element (EP);
Modify A.16(98/2)
The type Directory_Entry_Type represents a single item in a directory. These items can only be created {either }by the Get_Next_Entry procedure in this package{, or by generalized loop iteration on the return object of the Entries function in this package}. Information about the item can be obtained from the functions declared in this package. A default-initialized object of this type is invalid; objects returned from Get_Next_Entry {and by generalized loop iteration on the result of Entries} are valid.
Append after A.16(112/3)
type Directory_Listing is tagged limited private with Constant_Indexing => Current_Entry, Default_Iterator => Iterate, Iterator_Element => Directory_Entry_Type;
The type Directory_Listing contains the state of a directory search. A default-initialized Directory_Listing object has no entries available. The type Directory_Listing need finalization.
function Entries (Directory : in String; Pattern : in String; Filter : in Filter_Type := (others => True)) return Directory_Listing;
for generalized loop iteration (see 5.5.2), Entries returns an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each available directory entry in the return object. Entries starts a search in the directory named by Directory for entries matching Pattern and Filter. Pattern represents a pattern for matching file names. if Pattern is the null string, all items in the directory are matched; otherwise, the interpretation of Pattern is implementation-defined. Only items that match Filter will be returned. After a successful call on Entries, the return object may have entries available, but it may have no entries available if no files or directories match Pattern and Filter. The exception Name_Error is propagated if the string given by Directory does not identify an existing directory, or if Pattern does not allow the identification of any possible external file or directory. The exception Use_Error is propagated if the external environment does not support the searching of the directory with the given name (in the absence of Name_Error). when Start_Search propagates Name_Error or Use_Error, the return object will have no entries available.
function Iterate (Listing : Directory_Listing) return Directory_Iterators.Forward_Iterator'Class; Iterate returns a forward iterator object (see 5.5.1) that will generate a value for a loop parameter (see 5.5.2) designating each directory entry in Listing, starting with the first directory entry and moving the cursor as per the Next function. The iterator object needs finalization.
function Current_Entry (Entries : Directory_Listing; EP : Entry_Presence) return Directory_Entry_Type with Pre => Has_Element (EP);
If Entries is a default-initialized object or if EP is true and the current state of Entries has no entries available or is not involved in generalized loop iteration, then Program_Error is propagated. Otherwise, Current_Entry returns the Directory_Entry associated with the current state of the generalized iterator or container element iterator. It is implementation-defined as to whether the results returned by this subprogram are altered if the contents of the directory are altered during the current iteration of the loop (for example, by another program). The exception Use_Error is propagated if the external environment does not support continued searching of the directory represented by Entries.
Modify A.16(125/3)
Start_Search{,} [and ]Search{, and Entries} should raise Name_Error if Pattern is malformed, but not if it could represent a file in the directory but does not actually do so.
Modify A.16(125.a/3)
Implementation Advice: Directories.Start_Search{,} [and ] Directories.Search{, and Entries} should raise Name_Error for malformed patterns.
Modify A.17(3/2) "{with Ada.Iterator_Interfaces;}
package Ada.Environment_Variables is pragma Preelaborate(Environment_Variables);"
Append after A.17(8/3) the following
type Name_Value_Pair_Type is limited private;
type Environment_Variable_Listing is tagged limited private with Constant_Indexing => Current_Entry, Default_Iterator => Iterate, Iterator_Element => Name_Value_Pair_Type;
function All_Variables return Environment_Variable_Listing;
type Variable_Presence is new Boolean;
function Has_Element (VP : in Variable_Presence) return Boolean is (Boolean (VP));
package Environment_Variable_Iterators is new Ada.Iterator_Interfaces (Variable_Presence, Has_Element);
function Iterate (Listing : in Environment_Variable_Listing) return Environment_Variable_Iterators.Forward_Iterator'Class;
function Current_Variable (Variables : in Environment_Variable_Listing; VP : in Variable_Presence) return Name_Value_Pair with Pre => Has_Element (VP);
-- Operations on Name_Value_Pairs
function Name (Name_Value_Pair : in Name_Value_Pair_Type) return String;
function Value (Name_Value_Pair : in Name_Value_Pair_Type) return String;
Append after A.17(24/2)
type Name_Value_Pair_Type is limited private;
The type Name_Value_Pair_Type represents a single environment variable of the execution environment. These items can only be created by generalized loop iteration on the return object of the All_Variables function in this package}. Information about the item can be obtained from the functions declared in this package. A default-initialized object of this type is invalid; objects returned by generalized loop iteration on the result of Entries are valid.
type Environment_Variable_Listing is tagged limited private with Constant_Indexing => Current_Variable, Default_Iterator => Iterate, Iterator_Element => Name_Value_Pair_Type;
If the external execution environment supports environment variables, then for generalized loop iteration (see 5.5.2), an object of the type Environment_Variable_Listing can be an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each environment variable of the execution environment. Otherwise, Program_Error is propagated for generalized loop iteration involving the object. The type Environment_Variable_Listing contains the iteration state of the environment variables of the execution environment. A default-initialized Environment_Variable_Listing object represents all environment variables of the execution environment. The type Environment_Variable_Listing need finalization.
function All_Variables return Environment_Variable_Listing;
If the external execution environment supports environment variables, then for generalized loop iteration (see 5.5.2), All_Variables returns an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each environment variable of the execution environment. Otherwise, Program_Error is propagated.
function Iterate
(Variables : Environment_Variable_Listing)
return Environment_Variable_Iterators.Forward_Iterator'Class;
Iterate returns a forward iterator object (see 5.5.1) that will generate a value for a loop parameter (see 5.5.2) designating each environment variable in Variables, starting with the first environment variable and moving the cursor as per the Next function. The iterator object needs finalization.
function Current_Variable
(Variables : Environment_Variable_Listing; VP : Variable_Presence) return Name_Value_Pair with Pre => Has_Element (VP);
If VP is true and the current state of Variables is not involved in generalized loop iteration, then Program_Error is propagated. Otherwise, Current_Variable returns the Name_Value_Pair associated with the current state of the generalized iterator or container element iterator.
function Name (Name_Value_Pair : in Name_Value_Pair_Type) return String;
Returns the name of the environment variable represented by Name_Value_Pair.
function Value (Name_Value_Pair : in Name_Value_Pair_Type) return String; Returns the value of the environment variable represented by Name_Value_Pair.
Modify A.17(25/2)
It is a bounded error to call Value {if the call does not involve an object of type Name_Value_Pair_Type,} if more than one environment variable exists with the given name; the possible outcomes are that:
Modify A.17(28/2)
Making calls to the procedures Set or Clear concurrently with calls to any subprogram of package Environment_Variables, or to any instantiation of Iterate, {or during generalized loop iteration of an Environment_Variable_Listing object} results in erroneous execution.
!discussion
There were two identified approaches to provide this functionality. One was to have a function that returns an iterator_name that may be used as a generalized iterator. The second approach was to provide a function that returns an object of an iterable container type, that could be used to generate a container element iterator.
From the user's perspective, these two approaches would involve slightly different syntax, since generalized iterators use "in" in the iterator_specification while container element iterators use "of".
For example, a loop involving the first approach might look like;
for Dir_Entry in Ada.Directories.Entries ("/etc", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); end loop;
whereas for the second approach the same loop would be written as;
for Dir_Entry of Ada.Directories.Entries ("/etc", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); end loop;
To implement the first approach, the solution found required the use of a cursor type with an access discriminant and the Implicit_Dereference aspect to designate the corresponding Directory_Entry_Type for the loop. This was needed because Directory_Entry_Type is a limited type, and cannot be returned directly as object of the Ada.Iterator_Interfaces subprograms.
To implement this in the Ada.Directories package, the following would need to be added to the public part of the package.
with Ada.Iterator_Interfaces;
...
type Cursor
(Directory_Entry : not null access constant Directory_Entry_Type)
is private with
Implicit_Dereference => Directory_Entry;
function Has_Element (Directory_Entry : Cursor) return Boolean;
package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces (Cursor => Cursor, Has_Element => Has_Element);
function Entries (Directory : String; Pattern : String; Filter : Filter_Type := (others => True)) return Directory_Iterator_Interfaces.Forward_Iterator'Class;
It was generally felt that the use of the Implicit_Dereference aspect here was viewed as being a bit of a trick. It would be ideal if a solution could be found that was less "tricky". Also, having to expose a public type with an access discriminant was seen as being undesirable.
The second approach eliminated these concerns, because there is no need to involve access discriminants or the Implicit_Dererence aspect. In addition, the second approach is more flexible because it allows the user to write both forms of loops. The second form of the loop above could be written as;
declare Listing : Ada.Directories.Directory_Listing := Ada.Directories.Entries ("/etc", "*.txt"); begin for Dir_Entry in Listing.Iterate loop Put_Line (Directories.Simple_Name (Directories.Current_Entry (Entries => Listing, EP => Dir_Entry))); end loop; end;
It was also considered whether the container and the iterator type should be the same type. This seemed more confusing and would have required exposing more of the iterator interface in the public part of the package. It also would have been less clear whether one could restart an iteration on the same container object a second time.
It was considered whether reverse iteration through the containers should be supported. Since the existing iteration support for Ada.Directories did not support reverse iteration, it was felt that there would be insufficient need for this capability for generalized loop iteration.
A similar approach is applied to Ada.Environment_Variables.
!ACATS test
** TBD.
!appendix

From: Brad Moore
Sent: Saturday, October 1, 2011  1:37 PM

!topic Ada 2012 Iterators for Directory Searches in Ada.Directories
!reference Ada 2012 RM A.16 (32/2, 36/2)
!from /Brad Moore 11-10-01/
!keywords Iterators
!discussion

Ada 2012 introduces new iterator syntax for containers. A directory
can be viewed as a container for files. The Ada.Directories package
provides the Search related subprograms to iterate through the files
in a directory. It seems we ought to be able to use the new iterator
syntax to also iterate through the files in a directory. This would
be more convenient, and express the iteration in a more natural way,
similar to how the new syntax is helpful for iterating through containers.

****************************************************************

From: Brad Moore
Sent: Saturday, October 1, 2011  2:28 PM

!topic  Ada 2012 iterators for environment variables
!reference Ada 2012 RM A.17 (8/3)
!from /Brad Moore 11-10-01/
!keywords Iterators, Environment Variables
!discussion

Ada 2012 introduces new syntax for iterating through containers.
The package Ada.Environment_Variables has a subprogram called Iterate
that is used to iterate through the environment variables. It seems we ought
to be able to use the new iterator syntax to iterate through environment
variables as well. This likely would be more convenient and provide a more
natural means of expressing this functionality similar to how the new syntax
benefits iterating through containers.

****************************************************************

From: Brad Moore
Sent: Tuesday, September 29, 2015  9:15 PM

I have been working on solutions to this AI, and have encountered a bit of a
dilemma.

I started looking at Ada.Directories, and found I was able to take the existing
GNAT implementation, and get a working prototype by adding just the following to
the public part of the package specification.


    type Cursor
      (Directory_Entry : not null access constant Directory_Entry_Type)
    is private
    with
       Implicit_Dereference => Directory_Entry;

    function Has_Element (Directory_Entry : Cursor) return Boolean;

    package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces
      (Cursor      => Cursor,
       Has_Element => Has_Element);

    function Entries
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
       return Directory_Iterator_Interfaces.Forward_Iterator'Class;


I found the Implicit_Dereference aspect helpful here, as it allowed me to create
a cursor that can designate the limited type Directory_Entry_Type.

This allowed me to write:

for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
    Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;

Where I have defined an iterator_name, rather than an iterable_name (see 5.5.2).

Other than using the Rosen Trick, the implementation was quite easy and uses
existing library functions defined by the standard.

Looking at the meeting minutes, it was suggested that I look at having the
Entries function return an array of Directory_Entry_Types, rather than an
iterator object.  I suspect this is partly because noone had yet considered
using the Implicit_Dereference aspect as I have.

The main difference to the programmer between what was suggested and what I
implemented would be that the user would need to use "of" instead of "in" in the
loop iterator specification.

ie.

for Dir_Entry of Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
   Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;

Not a huge difference. But there are some others. Returning an array could be a
problem if the directory consisted of many files. A few thousand files might
cause stack overflow.

The solution I came up with however scales to any number of files because the
object being returned is just an iterator object, not an array. The iterator
object only contains a single Directory_Entry_Type object component at a time.

Another difference is that I think perhaps it is better to avoid "implicit"
containers. There is no array object that the programmer can see. I think using
iterable_names are good to use when you have an explicit container object
declaration in the programmers code, but otherwise, an iterator_name seems more
appropriate when there is no such explicit container.

On the other hand, this (and the environment variables package which could be
handled the same way) would be the first standard containers where the new Ada
2012 iterators are designed mainly for "in" loops rather than "of" loops.

I think considering all the above, I am still leaning towards what I have, but
wanted to get others opinions.

Which would be better?
   Use "in" instead of "of" in the loop iterators, returning an iterator object
   rather than an array? Or the other way round?


If it is helpful to look at the rest of the implementation I came up
with, here is what I added to the private part of the specification:

    type Cursor
      (Directory_Entry : not null access constant Directory_Entry_Type) is
       record
          Done : Boolean;
          --  Control : Reference_Control_Type :=
          --    raise Program_Error with "uninitialized reference";
          --  The RM says, "The default initialization of an object of
          --  type Constant_Reference_Type or Reference_Type propagates
          --  Program_Error."
       end record;

    function Has_Element (Directory_Entry : Cursor) return Boolean is
       (not Directory_Entry.Done);

    type Directory_Iterator;

    type Holder (Reference : access Directory_Iterator) is limited null record;

    type Directory_Iterator is limited new
      Directory_Iterator_Interfaces.Forward_Iterator with
       record
          -- Rosen trick needed so that First and Next can
          -- modify the iterator object
          Self : Holder (Directory_Iterator'Access);
          Search : Search_Type;
          Directory_Entry : aliased Directory_Entry_Type;
    end record;

    overriding
    function First (Object : Directory_Iterator) return Cursor;

    overriding
    function Next
      (Object : Directory_Iterator;
       Value  : Cursor) return  Cursor;

and in the body, I added....

    function Entries
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
       return Directory_Iterator_Interfaces.Forward_Iterator'Class is
    begin
       return It : Directory_Iterator do
          It.Done := False;

          Start_Search
            (Search    => It.Search,
             Directory => Directory,
             Pattern   => Pattern,
             Filter    => Filter);

       end return;

    end Entries;

    function First
      (Object : Directory_Iterator) return Cursor
    is
       Result : Cursor
         (Directory_Entry => Object.Directory_Entry'Unchecked_Access);
    begin
       Get_Next_Entry
         (Search          => Object.Self.Reference.Search,
          Directory_Entry => Object.Self.Reference.Directory_Entry);

       return Result;
    end First;

    function Next
      (Object : Directory_Iterator;
       Value  : Cursor) return Cursor
    is
       Result : Cursor
         (Directory_Entry => Object.Directory_Entry'Unchecked_Access);
    begin

       if More_Entries (Search => Object.Self.Reference.Search) then
          Get_Next_Entry
            (Search          => Object.Self.Reference.Search,
             Directory_Entry => Object.Self.Reference.Directory_Entry);
       else
          End_Search (Search => Object.Self.Reference.Search);
          Result.Done := True;
       end if;

       return Result;
    end Next;

****************************************************************

From: Ed Schonberg
Sent: Sunday, October 4, 2015  9:51 AM

> Which would be better?
>  Use "in" instead of "of" in the loop iterators, returning an iterator
>  object rather than an array? Or the other way round?

I think you make a good argument that using an iterator object is preferable to
constructing a temporary array.  This points to a construct that has become very
popular in other languages:  the generator.  Of course we can write generators
in Ada, but there is no single syntax that makes the idiom immediately
available.  Maybe some new aspect for a protected function?  It would be great
to have a simple way to describe a function where successive calls generate a
stream of values from some collection, without building the collection
explicitly.  Language design invited…

****************************************************************

From: Tucker Taft
Sent: Sunday, October 4, 2015  8:34 PM

> I think you make a good argument that using an iterator object is
> preferable to constructing a temporary array.  This points to a
> construct that has become very popular in other languages:  the generator.
> Of course we can write generators in Ada, but there is no single syntax that
> makes the idiom immediately available.

I would think that an iterator object, or an iterable collection object,
provides what you want.  Can you give an example of what a generator would do
that is not accomplishable with the First/Next sequence of iterators?

> ... Maybe some new aspect for a protected function?

I am not sure I understand why a protected function would be involved.

> ... It would be great to have
> a simple way to describe a function where successive calls generate a
> stream of values from some collection, without building the collection
> explicitly.  Language design invited…

I would certainly hope we could use one of the existing syntaxes "for I in XXX
loop" or "for I of YYY loop" to accomplish this.  The interesting question is
what these would expand into for a "generator" vs. an "iterator" (hard for me
since I am unsure of the difference).

****************************************************************

From: Randy Brukardt
Sent: Monday, October 5, 2015  6:08 PM

...
> I would think that an iterator object, or an iterable collection
> object, provides what you want.  Can you give an example of what a
> generator would do that is not accomplishable with the First/Next
> sequence of iterators?

I was equally puzzled by Ed's message. The only thing I can think of is
generating objects without some sort of Cursor (pointer) involved. But that only
works if:
(A) Objects in the stream are not modifiable, as in a functional language.
(You have to have some sort of access to the object in order to modify it within
the loop.)
(B) The order that the objects are to be returned is inherent in the data
structure. (The cursor provides a "next" location; it's hard to imagine many
uses where that isn't necessary.)

And even then, the implementation of the object stream probably will need some
sort of pointer, so using it as a cursor isn't a real issue.

In the vast majority of cases, you're going to need the operations of a forward
iterator, and since there are only two operations to implement, it's hard to
imagine how it could be made much simpler.

****************************************************************

From: Ed Schonberg
Sent: Tuesday, October 6, 2015  8:42 AM

What generators provide is lazy evaluation, when you are searching over a
potentially unbounded collection and you want to generate its values one by one.
Think of a search over prime numbers, or a crossword puzzle solver that will
show you words of 9 letters when the middle one is a Z!  You don't need to
precompute the set, you want to have a construct with local memory that embodies
the computation to go from one element of the set to the next.   In Brad's
example, the set of values exists in the directory so you can use cursors, but
that may not be the case: in GPS for example there is a typing completion
algorithm that generates dynamically suggestions based on keystrokes so far.  It
is not possible to precompute a set of completions in advance,  and what you
need is to encapsulate that set in a generator that will produce successive
candidates on demand.

****************************************************************

From: Tucker Taft
Sent: Tuesday, October 6, 2015  10:03 AM

> ... It is not possible to precompute a set of completions in advance,
> and what you need is to encapsulate that set in a generator that will produce
> successive candidates on demand.

Is there something that requires an iterator to be static?  Couldn't the call on
"Next" compute or retrieve the next value?

****************************************************************

From: Ed Schonberg
Sent: Tuesday, October 6, 2015  10:21 AM

Yes, but either the caller saves the previous value and Next takes it as an
argument, or else Next needs local memory, which is why coroutines come up in
these discussions.  This is why I mentioned protected types earlier: the
simplest realization of that model  in current Ada is a protected object with
one function and some local storage.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, October 6, 2015  11:39 AM

> Yes, but either the caller saves the previous value and Next takes it
> as an argument,

Which of course is exactly the model of Ada's iterators. For this case, the
"cursor" would be whatever is needed to understand the previous value.

> or else Next needs local memory,

The iterator object provides the local state. (Next having two parameters.)

> which is why coroutines come up in these discussions.  This is why I
> mentioned protected types earlier: the simplest realization of that
> model  in current Ada is a protected object with one function and some
> local storage.

Actually, the existing iterators map it pretty well so far as I can tell. I
recall Bob asking about lazy evaluation of a list during the early design, and
the intent was that they could be used that way. Perhaps a bit more complex than
a single function, but not particularly bad.

And of course, if you don't want the for loop syntax, you can just use a
function and some local state. I don't see anything else that the language could
do to help you in that case (I don't think Ada is getting lazy evaluation or
coroutines anytime in the near future!).

****************************************************************

From: Tucker Taft
Sent: Monday, October 5, 2015  7:30 PM

...
> Other than using the Rosen Trick, the implementation was quite easy
> and uses existing library functions defined by the standard.

What happens if you copy the value of a cursor and save it?  It seems the cursor
always points at the "current" directory entry.

> Looking at the meeting minutes, it was suggested that I look at having
> the Entries function return an array of Directory_Entry_Types, rather
> than an iterator object.  I suspect this is partly because noone had
> yet considered using the Implicit_Dereference aspect as I have.
>
> The main difference to the programmer between what was suggested and
> what I implemented would be that the user would need to use "of"
> instead of "in" in the loop iterator specification.
>
> ie.
>
> for Dir_Entry of Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
>    Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;
>
> Not a huge difference. But there are some others. Returning an array
> could be a problem if the directory consisted of many files. A few
> thousand files might cause stack overflow.

Do you mean secondary stack overflow, because these objects would almost
certainly not reside on the "primary" stack?  That seems relatively unlikely.

Alternatively, did you consider creating a function that returns a (conceptual)
directory-entry "container" rather than an actual array, which could be
allocated on the heap, and use a cursor object containing a reference-counted
pointer to the container, and an index into the container?

> The solution I came up with however scales to any number of files
> because the object being returned is just an iterator object, not an
> array. The iterator object only contains a single Directory_Entry_Type
> object component at a time.

True, but then the "cursor" is a bit of a misnomer, as it is really just a level
of indirection.

> Another difference is that I think perhaps it is better to avoid
> "implicit" containers. There is no array object that the programmer
> can see.

I had presumed that the function would be returning a container that was
indexable, so in what sense is that "implicit"?  It is a temporary, that is
true, though presumably it could be built in one place and iterated multiple
times.

> ... I think using iterable_names are good to use when you have an
> explicit container object declaration in the programmers code, but
> otherwise, an iterator_name seems more appropriate when there is no
> such explicit container.
>
> On the other hand, this (and the environment variables package which
> could be handled the same way) would be the first standard containers
> where the new Ada 2012 iterators are designed mainly for "in" loops rather than "of" loops.
>
> I think considering all the above, I am still leaning towards what I
> have, but wanted to get others opinions.
>
> Which would be better?
>    Use "in" instead of "of" in the loop iterators, returning an
> iterator object rather than an array? Or the other way round? ...

I think "in" or "or" is not that big of an issue.  The more important question
might be what are the semantics of saving the (temporary) container, or saving a
cursor?  Does a cursor end up as a dangling reference.  Do all cursor values
point at the same object?

****************************************************************

From: Randy Brukardt
Sent: Monday, October 4, 2015  6:19 PM

...
> The main difference to the programmer between what was suggested and
> what I implemented would be that the user would need to use "of"
> instead of "in" in the loop iterator specification.

Actually, the main difference is that the implementation-defined behavior when a
directory is altered is avoided. (See A.16(110/3).) The array would clearly be a
snapshot of the directory at the time it is taken, so it wouldn't change
afterwards.

Also, as Tucker notes, copying indexes into the array would be harmless.
Copying one of your cursors would most likely leave it being a dangling pointer
after the completion of the iteration. Sounds like more erroneous execution
possibilities (which I'd prefer to avoid).

I could see ways to avoid cursors becoming erroneous if someone copies them, but
they all boil down to making an "implicit container", which you say you don't
want to do.

****************************************************************

From: Brad Moore
Sent: Thursday, October 8, 2015  1:23 AM

>> This allowed me to write:
>>
>> for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
>>     Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;
>>
>
> What happens if you copy the value of a cursor and save it?  It seems
> the cursor always points at the "current" directory entry.
>

I was kind of hoping that the accessibility rules would somehow prevent
squirreling a temporary access value from an inner scope to an outer scope. The
cursor's discriminant is an access to a Directory_Entry_Type component of the
iterator object. I tried this with GNAT however, and it didn't complain. Copying
inside the loop body should be OK since any copy would be expected to designate
the same "current" directory entry. Shouldn't the accessibility checks prevent
such copying to variables declared in a global scope?

declare
   DE : aliased Directories.Directory_Entry_Type;
   Squirrel : Directories.Cursor (DE'Access); begin

    for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
       Put_Line (Directories.Simple_Name (Dir_Entry));
       Squirrel := Directories.Cursor'(Dir_Entry);
    end loop;
end;

****************************************************************

From: Randy Brukardt
Sent: Thursday, October 8, 2015  1:07 PM

...
> Shouldn't the accessibility checks prevent such copying to variables
> declared in a global scope?
>
> declare
>    DE : aliased Directories.Directory_Entry_Type;
>    Squirrel : Directories.Cursor (DE'Access); begin
>
>     for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
>        Put_Line (Directories.Simple_Name (Dir_Entry));
>        Squirrel := Directories.Cursor'(Dir_Entry);
>     end loop;
> end;

The above is fine for accesibility (I think), but it should raise
Constraint_Error because you can't change the discriminant of Squirrel (thus the
discriminant check should fail). Try: (I didn't compile this)

declare
   type Outer is access all Directories.Directory_Entry_Type;
   Squirrel : Outer;
begin
     for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
        Put_Line (Directories.Simple_Name (Dir_Entry));
        Squirrel := Dir_Entry.D; -- D should be whatever the name of your discriminant is.
     end loop;
end;

...
>I was kind of hoping that the accessibility rules would somehow prevent squirreling
>a temporary access value from an inner scope to an outer scope. The cursor's
>discriminant is an access to a Directory_Entry_Type component of the iterator object.

They're supposed to do that. You shouldn't be able to copy the discriminant to
any named access type that is outside of the loop.

But Tucker's point is that you could squirrel them between iterations, since all
of the iterations belong to a single master. The good news is I can't figure out
how to write such a thing [if you add a block, you're at a different level];
maybe with a Holder container and a subprogram? Or maybe it's impossible and
Tucker is just throwing FUD.

****************************************************************

From: Tucker Taft
Sent: Thursday, October 8, 2015  4:40 PM

There are a couple of subtle issues here.  Not surprising because we are
tip-toeing very near the heart of darkness (3.10.2).  I think we are probably
OK, but convincing myself of that was not easy.  The loop parameter Dir_Entry is
of the Cursor type, and it is *initialized* by a call on First, so no constraint
check.  The subsequent iterations have Dir_Entry being *assigned* from the
result of Next, so here there *would* be a constraint check, presumably, and if
the access discriminant pointed elsewhere, then that should fail the
discriminant check.  So in fact, if you are going to use this approach, then the
access discriminant of the cursor object must *always* point to the same object.
A bit weird!

As far as squirreling away, I agree that the accessibility level of the access
discriminant's type should be the same as that of the loop parameter (per
3.10.2(12.5/3)), so GNAT should disallow the assignment to an access-type
variable outside the loop if you are assigning from the access discriminant
only, because of the required implicit or explicit conversion (per
4.6(24.17/4)); it should fail a discriminant check if you assign from the loop
parameter as a whole.  It should also disallow using the value of the loop
parameter, or the access discriminant thereof, as the initial value in an
initialized allocator for an access type declared outside of the loop, per
4.8(5.3/3, 10.1/3).

So this looks OK, but it shows some issues with GNAT's discriminant checks
(unless you are suppressing them, or you compiled but did not actually run the
code).  It will be interesting to see whether GNAT rejects Randy's example,
where you are assigning from the access discriminant itself.

After all of this, I am somewhat uncomfortable using this trick to provide an
iterator. Alternatively, we should bless this approach and recommend it for
other cases where you want to use the iterator syntax but not actually use the
cursor as an index into anything, but rather just use it as a level of
indirection into the iterator object, which has the current value stored within
it.   Calling it a "cursor" is of course a bit confusing then.
  It is more like a "handle" on the current object of the iteration, which is
  hidden somewhere within the iterator object itself (in this case) or something
  it references (in the container iterator case).

We are relying on the access discriminant to prevent squirreling things away
(thanks to the discriminant check), as well as giving us the implicit
dereference semantics.  I would be somewhat more comfortable if we hid the
access discriminant, and perhaps the whole "cursor" notion completely.

What would happen if we turned this into a container element iterator, by
changing "Entries" to be something that could follow "of" rather than "in"?
That is, "Entries" would return an iterable object that has a Constant_Indexing
aspect.  The Constant_Indexing aspect would return the "current" Directory_Entry
(or a const reference to the current Directory_Entry if we want to avoid copying
a large Directory_Entry), pretty much independent of the cursor value passed in.
The cursor could be effectively a Boolean value, indicating whether or not there
is another directory entry to be processed, so Has_Element could simply return
the Boolean value of the cursor, and the Constant_Indexing attribute could raise
an exception if passed a cursor with value False.

For example:

    type Entry_Presence is new Boolean; -- the "cursor" type
    function Has_Element(EP : Entry_Presence) return Boolean is (Boolean(EP));

    package Dir_Iterators is
      new Ada.Iterator_Interfaces(Entry_Presence, Has_Element);

    type Dir_Entry_Iterator is  --  The iterable container type
      limited new Dir_Iterators.Forward_Iterator
        with Constant_Indexing => Current_Entry,
             Default_Iterator => Entries,
             Iterator_Element => Dir_Entry;

    function Entries  --  Returns an iterable container
      (Dir_Name : String; Pattern : String)
       return Dir_Entry_Iterator;

    function First   --  One of the forward iterator operations
      (Entries: Dir_Entry_Iterator)
       return Entry_Presence;
    function Next    --  One of the forward iterator operations
      (Entries: Dir_Entry_Iterator; Prev : Entry_Presence)
       return Entry_Presence;

    function Current_Entry  --  "indexing" operation
      (Entries: Dir_Entry_Iterator; EP : Entry_Presence)
       return Dir_Entry
      with Pre => Has_Element(EP);
       --  or could return an object that has an access discrim
       --  e.g.:
       --     type Dir_Ent_Ref(DE : not null access constant Dir_Entry) ...


    for Dir_Ent of Entries("/homes/stt", "*.txt") loop
      ...
    end loop;

This would preserve the model of a directory as a vector of directory entries,
while still allowing us to implement the iteration without building up that
vector explicitly.  It also eliminates the visible use of a reference object as
a cursor, and the concerns about an attempt at squirreling away the cursor
causing a run-time constraint error or a potentially confusing accessibility
error.

****************************************************************

From: Tucker Taft
Sent: Thursday, October 8, 2015  5:00 PM

I think I might have gotten myself confused with the "Default_Iterator" aspect.
Stay tuned for another version of the actual source code (I might actually try
to compile it next time ;-).  I think the basic idea should still work, allowing
us to use "of" rather than "in" and avoid some of the accessibility check
issues.  I just need to get my "iterable container" and "default iterator" and
"iterator element" aspects all straightened out...

****************************************************************

From: Tucker Taft
Sent: Thursday, October 8, 2015  5:13 PM

OK, the change is to make the Default_Iterator aspect denote an identity
function, which takes the Dir_Entry_Iterator and returns it, since the
Dir_Entry_Iterator is acting as both our "iterable container type" and as an
"iterator type."  Hence:

    type Dir_Entry_Iterator is  --  The iterable container type
      new Dir_Iterators.Forward_Iterator
        with Constant_Indexing => Current_Entry,
             Default_Iterator => Identity,
             Iterator_Element => Dir_Entry;

    function Identity(Iter: Dir_Entry_Iterator) return Dir_Entry_Iterator is (Iter);

Note that because we wanted to define the body of Identity using a simple
expression function, rather than an aggregate, we had to change
Dir_Entry_Iterator to be a non-limited type.  Everything but the
Default_Iterator and limitedness of Dir_Entry_Iterator remains the same.

Sorry for the confusion.

****************************************************************

From: Brad Moore
Sent: Thursday, October 8, 2015  8:00 PM

...
> As far as squirreling away, I agree that the accessibility level of
> the access discriminant's type should be the same as that of the loop
> parameter (per 3.10.2(12.5/3)), so GNAT should disallow the assignment
> to an access-type variable outside the loop if you are assigning from
> the access discriminant only, because of the required implicit or
> explicit conversion (per 4.6(24.17/4)); it should fail a discriminant
> check if you assign from the loop parameter as a whole.  It should
> also disallow using the value of the loop parameter, or the access
> discriminant thereof, as the initial value in an initialized allocator
> for an access type declared outside of the loop, per 4.8(5.3/3, 10.1/3).
>
> So this looks OK, but it shows some issues with GNAT's discriminant
> checks (unless you are suppressing them, or you compiled but did not
> actually run the code).

Sorry, it was late last night, when I was trying this out.
I compiled but did not run the code.
When I run it today, I see the Constraint_Error being raised, as expected.

****************************************************************

From: Brad Moore
Sent: Thursday, October 8, 2015  11:46 PM

I took your suggestion and ran with it, but found there were some further tweaks
required.

In particular, I found that I had to change Dir_Entry_Iterator back to being a
limited type. This is because I needed to be able to modify an "in" parameter
(in calls such as First, and Next), so I had to use the Rosen trick to do this.
However, this required the type to be limited. As a result of making it limited,
I had to involve two separate objects, the Container object and the Iterator
object. Both are still the same type, (and maybe that's confusing). Because
Identity needs to return a limited type, it cannot just return its "Iter"
parameter. It has to construct a new return object from the "Iter" parameter. I
decided to put the storage for the Directory_Entry in the Iterator object, but
the Current_Entry function receives the Container object, not the Iterator
object, and needs to ultimately get to the Iterator object, so I had to store an
extra access value from the container to access the iterator object. Only the
container object has this access value. The iterator object sets its own value
for this component to null.

All in all, the implementation was a bit tricky to get working, but it does seem
to work. I have it compiling and executing with proper results in GNAT.

The new content added to the public part of the Ada.Directories package is;

    type Entry_Presence is new Boolean; -- the "cursor" type

    function Has_Element
      (EP : Entry_Presence) return Boolean is (Boolean (EP));

    package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces
      (Cursor      => Entry_Presence,
       Has_Element => Has_Element);

     package Dir_Iterators is
       new Ada.Iterator_Interfaces (Entry_Presence, Has_Element);

     type Dir_Entry_Iterator is limited --  The iterable container type
      new Dir_Iterators.Forward_Iterator with private
        with Constant_Indexing => Current_Entry,
             Default_Iterator => Identity,
             Iterator_Element => Directory_Entry_Type;

    overriding
    function First (Iter : Dir_Entry_Iterator) return Entry_Presence;

    overriding
    function Next
      (Iter : Dir_Entry_Iterator;
       Value  : Entry_Presence) return  Entry_Presence;

    function Current_Entry  --  "indexing" operation
      (Entries : Dir_Entry_Iterator; EP : Entry_Presence)
       return Directory_Entry_Type
      with Pre => Has_Element (EP);

    function Identity (Iter : Dir_Entry_Iterator) return Dir_Entry_Iterator;

    function Entries  --  Returns an iterable container
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
        return Dir_Entry_Iterator;

I think I agree with you that this is probably a better approach than the "in"
syntax version.

****************************************************************

From: Brad Moore
Sent: Saturday, October 10, 2015  12:26 PM

I was able to go further and hide the "noise" parts of the iterator mechanism in
the private part of the package. I ended up creating two separate types to
distinguish the container from the iterator. The Container is a type called
"Directory_Listing", and the Iterator type is defined in the private part of the
package. I was able to move the "Identity" function and the "Current_Entry"
function into the private part of the package as well, since the current
freezing rules allowed me to do this.

I think in this case, it a nice feature because those two functions are not
intended to be called by programmers, and are only needed to support the
iterator mechanism. Thus, I was able to reduce the additions needed for to the
public part of the package to just;

    private with Ada.Iterator_Iterfaces;
    ...

    type Directory_Listing is tagged limited private
       with Constant_Indexing => Current_Entry,
            Default_Iterator => Identity,
            Iterator_Element => Directory_Entry_Type;

    function Entries
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
        return Directory_Listing;

****************************************************************

From: Brad Moore
Sent: Saturday, October 10, 2015  5:59 PM

Here is my writeup for the AI12-0009-1 [This is version /02 of the AI - Editor.]

****************************************************************

From: Tucker Taft
Sent: Saturday, October 10, 2015  8:44 PM

...
> I think in this case, it a nice feature because those two functions
> are not intended to be called by programmers, and are only needed to
> support the iterator mechanism. Thus, I was able to reduce the
> additions needed for to the public part of the package to just;
>
>     private with Ada.Iterator_Iterfaces;
>     ...
>
>     type Directory_Listing is tagged limited private
>        with Constant_Indexing => Current_Entry,
>             Default_Iterator => Identity,
>             Iterator_Element => Directory_Entry_Type;

I don't think you can refer to a declaration that occurs in the private part in
an aspect, if the aspect specification is in the visible part.  See
13.1.1(11/3):

   "The usage names in an aspect_definition are not resolved at the point of the
    associated declaration, but rather are resolved at the end of the
    immediately enclosing declaration list."

****************************************************************

From: Randy Brukardt
Sent: Saturday, October 4, 2015  9:53 PM

Yup. I just finished looking that up to write essentially the same message that
you did. But I see you beat me to it, between the time I read Brad's message on
my phone and the time I arrived in the office to answer it (but mainly to
reprogram my handheld GPS for the northeast).

****************************************************************

From: Bob Duff
Sent: Sunday, October 11, 2015  6:11 AM

> I don't think you can refer to a declaration that occurs in the
> private part in an aspect, if the aspect specification is in the visible part.
> See 13.1.1(11/3):

You can in older versions of GNAT.  This bug was fixed fairly recently.

****************************************************************

From: Randy Brukardt
Sent: Monday, October 12, 2015  3:11 PM

There really ought to be an ACATS test for that. It's one of the higher priority
untested things; any volunteers??

****************************************************************

From: Brad Moore
Sent: Sunday, October 11, 2015  12:22 PM

> I don't think you can refer to a declaration that occurs in the
> private part in an aspect, if the aspect specification is in the visible part.
> See 13.1.1(11/3):
>
>    "The usage names in an aspect_definition are not resolved at the
> point of the associated declaration, but rather are resolved at the
> end of the immediately enclosing declaration list."
>

That's somewhat unfortunate as it means having to pull in more declarations from
the private part that are mostly of no concern to the programmer, and having to
deal with unnecessary use cases.

For example for the function,
   function Current_Entry
      (Entries : Directory_Listing; EP : Entry_Presence)
       return Directory_Entry_Type
      with Pre => Has_Element (EP);

we have to now deal with the possibility that a user might call this function
explicitly with EP having a value of True, while the state of the
Directory_Listing container might be in a state where the current directory
entry is not present.

It makes me think that there needs to be some capability to declare that a
subprogram is not explicitly callable by the programmer, and only callable by
the implementation.

I'm thinking of a User_Callable aspect that can be applied to the subprogram as
in;

  function Current_Entry
    (Entries : Directory_Listing; EP : Entry_Presence)
      return Directory_Entry_Type
    with Pre => Has_Element (EP),
         User_Callable => False;

This would hopefully would mean that less semantics and wording for such a
function would be needed in the RM. I think this helps also to separate the real
public interface from the bits and pieces that are not intended to be called by
user.

****************************************************************

From: Brad Moore
Sent: Sunday, October 11, 2015  3:09 PM

Here is an updated version to replace what I submitted yesterday. [This is
version /03 of the AI - Editor.]

This version eliminates the "bug" where I was assuming I could defer the
Contant_Indexing function and Default_Iterator functions to the private part
of the packages.

This version pulls the necessary declarations from the private part back into
the public part. I have tested this implementation for the Ada.Directories
package, and have a working implementation.

I also renamed Identity to Iterate, as Identity is no longer true and Iterate
is more consistent with the naming we use on other existing containers.

For now, disregard my comment in my previous email about providing a
User_Callable aspect. I don't think it is warranted here.

I think we want to expose Iterate and Current_Entry as user callable. 
This allows one to write both forms of generalized iterator loops. 
(Using "in" or "of" in the loop iterator_specification)

****************************************************************

From: Randy Brukardt
Sent: Monday, October 12, 2015  3:43 PM

> Here is an updated version to replace what I submitted yesterday.
> This version eliminates the "bug" where I was assuming I could defer 
> the Contant_Indexing function and Default_Iterator functions to the 
> private part of the packages.

I corrected a couple of typos in this as I was posting it. Specifically,

...
> I also renamed Identity to Iterate, as Identity is no longer true and 
> Iterate is more consistent with the naming we use on other existing 
> containers.

The Default_Iterator aspect was still set to Identity in A.16(36/3). You
certainly meant that to be Iterate as well.

Also, !question should be !problem in an !Amendment class AI.

****************************************************************


Questions? Ask the ACAA Technical Agent