Version 1.8 of ai12s/ai12-0009-1.txt

Unformatted version of ai12s/ai12-0009-1.txt version 1.8
Other versions for file ai12s/ai12-0009-1.txt

!standard A.16(3/2)          16-06-07 AI12-0009-1/06
!standard A.16(36.1/3)
!standard A.16(98/2)
!standard A.16(112.1/3)
!standard A.16(125/3)
!standard A.17(3/2)
!standard A.17(8/3)
!standard A.17(23/3)
!standard A.17(25/2)
!standard A.17(28/2)
!class Amendment 11-11-08
!status work item 11-11-08
!status received 11-10-01
!priority Low
!difficulty Medium
!subject Iterators for Directories and Environment_Variables
!summary
Support for generalized loop iteration for Ada.Directories and Ada.Environment_Variables is added.
!problem
Ada.Directories and Ada.Environment_Variables contain iterator procedures with call-back procedures. There should be a version of the Ada 2012 iterators for these subprograms, so that users can write a loop.
!proposal
(See wording.)
!wording
Modify A.16(3/2) "with Ada.IO_Exceptions;
with Ada.Calendar;
{with Ada.Iterator_Interfaces;}
package Ada.Directories is"
Append after A.16(36/3) the following
-- Directory Iteration
type Directory_Listing is tagged limited private
with Constant_Indexing => Current_Entry,
Default_Iterator => Iterate, Iterator_Element => Directory_Entry_Type;
pragma Preelaborable_Initialization (Directory_Listing);
function Entries
(Directory : in String;
Pattern : in String; Filter : in Filter_Type := (others => True)) return Directory_Listing;
type Cursor is private; pragma Preelaborable_Initialization (Cursor);
function Has_Entry (Position : in Cursor) return Boolean;
package Directory_Iterators is new Ada.Iterator_Interfaces (Cursor, Has_Entry);
function Iterate (Listing : in Directory_Listing) return Directory_Iterators.Forward_Iterator'Class;
function Current_Entry (Entries : in Directory_Listing; Position : in Cursor) return Directory_Entry_Type with Pre => Has_Entry (Position);
Modify A.16(98/2)
The type Directory_Entry_Type represents a single item in a directory. These items can only be created {either }by the Get_Next_Entry procedure in this package{, or by generalized loop iteration on the return object of the Entries function in this package}. Information about the item can be obtained from the functions declared in this package. A default-initialized object of this type is invalid; objects returned from Get_Next_Entry {and by generalized loop iteration on the result of Entries} are valid.
Append after A.16(112/3)
type Directory_Listing is tagged limited private;
The type Directory_Listing contains the state of a directory search. An object of type Directory_Listing can be used in generalized loop iteration (see 5.5.2) to visit each available directory entry for a particular directory search. A default-initialized Directory_Listing object has no entries available. The type Directory_Listing needs finalization.
function Entries (Directory : in String; Pattern : in String; Filter : in Filter_Type := (others => True)) return Directory_Listing;
Returns an object that represents the result of a search in the directory named by Directory for entries matching Pattern and Filter. Pattern represents a pattern for matching file names. If Pattern is the null string, all items in the directory are matched; otherwise, the interpretation of Pattern is implementation-defined. Only items that match Filter will be returned. After a successful call on Entries, the return object may have entries available, but it may have no entries available if no files or directories match Pattern and Filter. The exception Name_Error is propagated if the string given by Directory does not identify an existing directory, or if Pattern does not allow the identification of any possible external file or directory. The exception Use_Error is propagated if the external environment does not support the searching of the directory with the given name (in the absence of Name_Error). When Start_Search propagates Name_Error or Use_Error, the return object will have no entries available.
type Cursor is private;
A cursor is a reference to a directory entry within a directory listing.
To Be Honest: There may not be an actual reference component needed, but the abstraction of a cursor into a directory listing is intuitive and is consistent with other container iterators in the standard.
function Has_Entry
(Position : in Cursor) return Boolean;
Returns True if Position designates a directory entry, and returns False otherwise.
function Iterate
(Listing : Directory_Listing)
return Directory_Iterators.Forward_Iterator'Class;
Iterate returns a forward iterator object (see 5.5.1) that will generate a value for a loop parameter (see 5.5.2) designating each directory entry in Listing, starting with the first directory entry and moving the cursor as per the Next function. The iterator object needs finalization.
function Current_Entry
(Entries : Directory_Listing;
Position : Cursor) return Directory_Entry_Type
with Pre => Has_Entry (Position);
If Entries is a default-initialized object then Program_Error is propagated. Otherwise, Current_Entry returns the Directory_Entry designated by Position when used with a generalized iterator or container element iterator. It is implementation-defined as to whether the results returned by this subprogram are altered if the contents of the directory are altered during the current iteration of the loop (for example, by another program). The exception Use_Error is propagated if the external environment does not support continued searching of the directory represented by Entries.
Modify A.16(125/3)
Start_Search{,} [and ]Search{, and Entries} should raise Name_Error if Pattern is malformed, but not if it could represent a file in the directory but does not actually do so.
Modify A.16(125.a/3)
Implementation Advice: Directories.Start_Search{,} [and ] Directories.Search{, and Directories.Entries} should raise Name_Error for malformed patterns.
Modify A.17(3/2) "{with Ada.Iterator_Interfaces;}
package Ada.Environment_Variables is pragma Preelaborate(Environment_Variables);"
Append after A.17(8/3) the following
package Name_Value_Pairs is
type Name_Value_Pair_Type is tagged limited private;
-- Operations on Name_Value_Pairs
function Name (Name_Value_Pair : in Name_Value_Pair_Type) return String;
function Value (Name_Value_Pair : in Name_Value_Pair_Type) return String;
private ... -- not specified by the languages end Name_Value_Pairs;
use Name_Value_Pairs;
type Environment_Variable_Listing is tagged limited private with Constant_Indexing => Current_Variable, Default_Iterator => Iterate, Iterator_Element => Name_Value_Pair_Type; pragma Preelaborable_Initialization (Environment_Variable_Listing);
function Environment return Environment_Variable_Listing;
type Cursor is private; pragma Preelaborable_Initialization (Cursor);
function Has_Environment_Variable (Position : in Cursor) return Boolean;
package Environment_Variable_Iterators is new Ada.Iterator_Interfaces (Cursor, Has_Environment_Variable);
function Iterate (Listing : in Environment_Variable_Listing) return Environment_Variable_Iterators.Forward_Iterator'Class;
function Current_Variable (Variables : in Environment_Variable_Listing; Position : in Cursor) return Name_Value_Pair_Type with Pre => Has_Environment_Variable (Position);
Append after A.17(24/2)
type Name_Value_Pair_Type is limited private;
The type Name_Value_Pair_Type represents a single environment variable of the execution environment. A default-initialized object of this type is invalid; objects returned by generalized loop iteration on the result of Entries are valid.
function Name (Name_Value_Pair : in Name_Value_Pair_Type) return String;
Returns the name of the environment variable represented by Name_Value_Pair.
function Value (Name_Value_Pair : in Name_Value_Pair_Type) return String; Returns the value of the environment variable represented by Name_Value_Pair.
type Environment_Variable_Listing is tagged limited private;
The type Environment_Variable_Listing contains the state of an environment variable search for the execution environment. An object of type Environment_Variable_Listing can be used in generalized loop iteration (see 5.5.2) to visit each environment variable from the external environment. A default-initialized Environment_Variable_Listing object represents all environment variables from the external environment. The type Environment_Variable_Listing needs finalization.
If the external execution environment does not support environment variables, then Program_Error is propagated for generalized loop iteration.
function Environment return Environment_Variable_Listing;
Returns an object that represents the state of an environment variable search for all environment variables from the execution environment. The exception Program_Error is propagated if the external execution environment does not support environment variables.
type Cursor is private;
A cursor is a reference to an environment variable name value pair within an environment variable listing.
To Be Honest: There may not be an actual reference component needed, but the abstraction of a cursor into an environment variable listing is intuitive and is consistent with other container iterators in the standard.
function Has_Environment_Variable
(Position : in Cursor) return Boolean;
Returns True if Position designates an environment variable name value pair, and returns False otherwise.
function Iterate
(Variables : Environment_Variable_Listing)
return Environment_Variable_Iterators.Forward_Iterator'Class;
Iterate returns a forward iterator object (see 5.5.1) that will generate a value for a loop parameter (see 5.5.2) designating each environment variable in Variables, starting with the first environment variable and moving the cursor as per the Next function. The iterator object needs finalization.
function Current_Variable
(Variables : Environment_Variable_Listing;
Position : Cursor) return Name_Value_Pair
with Pre => Has_Environment_Variable (Position);
Current_Variable returns the Name_Value_Pair designated by Position that is associated with the current state of the generalized iterator or container element iterator.
Modify A.17(25/2)
It is a bounded error to call Value {if the call does not involve an object of type Name_Value_Pair_Type,} if more than one environment variable exists with the given name; the possible outcomes are that:
!discussion
There were two identified approaches to provide this functionality. One was to have a function that returns an iterator_name that may be used as a generalized iterator. The second approach was to provide a function that returns an object of an iterable container type, that could be used to generate a container element iterator.
From the user's perspective, these two approaches would involve slightly different syntax, since generalized iterators use "in" in the iterator_specification while container element iterators use "of".
For example, a loop involving the first approach might look like;
for Dir_Entry in Ada.Directories.Entries ("/etc", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); end loop;
whereas for the second approach the same loop would be written as;
for Dir_Entry of Ada.Directories.Entries ("/etc", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); end loop;
To implement the first approach, the solution found required the use of a cursor type with an access discriminant and the Implicit_Dereference aspect to designate the corresponding Directory_Entry_Type for the loop. This was needed because Directory_Entry_Type is a limited type, and cannot be returned directly as object of the Ada.Iterator_Interfaces subprograms.
To implement this in the Ada.Directories package, the following would need to be added to the public part of the package.
with Ada.Iterator_Interfaces;
...
type Cursor
(Directory_Entry : not null access constant Directory_Entry_Type)
is private with
Implicit_Dereference => Directory_Entry;
function Has_Element (Directory_Entry : Cursor) return Boolean;
package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces (Cursor => Cursor, Has_Element => Has_Element);
function Entries (Directory : String; Pattern : String; Filter : Filter_Type := (others => True)) return Directory_Iterator_Interfaces.Forward_Iterator'Class;
It was generally felt that the use of the Implicit_Dereference aspect here was viewed as being a bit of a trick. It would be ideal if a solution could be found that was less "tricky". Also, having to expose a public type with an access discriminant was seen as being undesirable.
The second approach eliminated these concerns, because there is no need to involve access discriminants or the Implicit_Dererence aspect. In addition, the second approach is more flexible because it allows the user to write both forms of loops. The second form of the loop above could be written as;
declare Listing : Ada.Directories.Directory_Listing := Ada.Directories.Entries ("/etc", "*.txt"); begin for Dir_Entry in Listing.Iterate loop Put_Line (Directories.Simple_Name (Directories.Current_Entry (Entries => Listing, EP => Dir_Entry))); end loop; end;
It was also considered whether the container and the iterator type should be the same type. This seemed more confusing and would have required exposing more of the iterator interface in the public part of the package. It also would have been less clear whether one could restart an iteration on the same container object a second time.
It was considered whether reverse iteration through the containers should be supported. Since the existing iteration support for Ada.Directories did not support reverse iteration, it was felt that there would be insufficient need for this capability for generalized loop iteration.
The overall approach taken for Ada.Environment_Variables is similar to the one taken for Ada.Directories except that the type Name_Value_Pair_Type was made to be a tagged type to improve usability by allowing object prefix notation.
for Pair of Ada.Environment_Variables.Environment loops Put_Line (Pair.Name & "=" & Pair.Value); end loop;
rather than;
for Pair of Ada.Environment_Variables.Environment loops Put_Line (Ada.Environment_Variables.Name (Pair) & "=" & Ada.Environment_Variables.Value (Pair)); end loop;
Making Name_Value_Pair_Type tagged however, meant having to declare the type in a nested package, to avoid problems with having the call Current_Variable dispatching on more than one type, which is illegal.
It would have been nice to do the same for Directory_Entry_Type, and make it a tagged type but that type already existed and changing that to be a tagged type would likely break backwards incompatibility in some way.
!ACATS test
** TBD.
!appendix

From: Brad Moore
Sent: Saturday, October 1, 2011  1:37 PM

!topic Ada 2012 Iterators for Directory Searches in Ada.Directories
!reference Ada 2012 RM A.16 (32/2, 36/2)
!from /Brad Moore 11-10-01/
!keywords Iterators
!discussion

Ada 2012 introduces new iterator syntax for containers. A directory
can be viewed as a container for files. The Ada.Directories package
provides the Search related subprograms to iterate through the files
in a directory. It seems we ought to be able to use the new iterator
syntax to also iterate through the files in a directory. This would
be more convenient, and express the iteration in a more natural way,
similar to how the new syntax is helpful for iterating through containers.

****************************************************************

From: Brad Moore
Sent: Saturday, October 1, 2011  2:28 PM

!topic  Ada 2012 iterators for environment variables
!reference Ada 2012 RM A.17 (8/3)
!from /Brad Moore 11-10-01/
!keywords Iterators, Environment Variables
!discussion

Ada 2012 introduces new syntax for iterating through containers.
The package Ada.Environment_Variables has a subprogram called Iterate
that is used to iterate through the environment variables. It seems we ought
to be able to use the new iterator syntax to iterate through environment
variables as well. This likely would be more convenient and provide a more
natural means of expressing this functionality similar to how the new syntax
benefits iterating through containers.

****************************************************************

From: Brad Moore
Sent: Tuesday, September 29, 2015  9:15 PM

I have been working on solutions to this AI, and have encountered a bit of a
dilemma.

I started looking at Ada.Directories, and found I was able to take the existing
GNAT implementation, and get a working prototype by adding just the following to
the public part of the package specification.


    type Cursor
      (Directory_Entry : not null access constant Directory_Entry_Type)
    is private
    with
       Implicit_Dereference => Directory_Entry;

    function Has_Element (Directory_Entry : Cursor) return Boolean;

    package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces
      (Cursor      => Cursor,
       Has_Element => Has_Element);

    function Entries
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
       return Directory_Iterator_Interfaces.Forward_Iterator'Class;


I found the Implicit_Dereference aspect helpful here, as it allowed me to create
a cursor that can designate the limited type Directory_Entry_Type.

This allowed me to write:

for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
    Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;

Where I have defined an iterator_name, rather than an iterable_name (see 5.5.2).

Other than using the Rosen Trick, the implementation was quite easy and uses
existing library functions defined by the standard.

Looking at the meeting minutes, it was suggested that I look at having the
Entries function return an array of Directory_Entry_Types, rather than an
iterator object.  I suspect this is partly because noone had yet considered
using the Implicit_Dereference aspect as I have.

The main difference to the programmer between what was suggested and what I
implemented would be that the user would need to use "of" instead of "in" in the
loop iterator specification.

ie.

for Dir_Entry of Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
   Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;

Not a huge difference. But there are some others. Returning an array could be a
problem if the directory consisted of many files. A few thousand files might
cause stack overflow.

The solution I came up with however scales to any number of files because the
object being returned is just an iterator object, not an array. The iterator
object only contains a single Directory_Entry_Type object component at a time.

Another difference is that I think perhaps it is better to avoid "implicit"
containers. There is no array object that the programmer can see. I think using
iterable_names are good to use when you have an explicit container object
declaration in the programmers code, but otherwise, an iterator_name seems more
appropriate when there is no such explicit container.

On the other hand, this (and the environment variables package which could be
handled the same way) would be the first standard containers where the new Ada
2012 iterators are designed mainly for "in" loops rather than "of" loops.

I think considering all the above, I am still leaning towards what I have, but
wanted to get others opinions.

Which would be better?
   Use "in" instead of "of" in the loop iterators, returning an iterator object
   rather than an array? Or the other way round?


If it is helpful to look at the rest of the implementation I came up
with, here is what I added to the private part of the specification:

    type Cursor
      (Directory_Entry : not null access constant Directory_Entry_Type) is
       record
          Done : Boolean;
          --  Control : Reference_Control_Type :=
          --    raise Program_Error with "uninitialized reference";
          --  The RM says, "The default initialization of an object of
          --  type Constant_Reference_Type or Reference_Type propagates
          --  Program_Error."
       end record;

    function Has_Element (Directory_Entry : Cursor) return Boolean is
       (not Directory_Entry.Done);

    type Directory_Iterator;

    type Holder (Reference : access Directory_Iterator) is limited null record;

    type Directory_Iterator is limited new
      Directory_Iterator_Interfaces.Forward_Iterator with
       record
          -- Rosen trick needed so that First and Next can
          -- modify the iterator object
          Self : Holder (Directory_Iterator'Access);
          Search : Search_Type;
          Directory_Entry : aliased Directory_Entry_Type;
    end record;

    overriding
    function First (Object : Directory_Iterator) return Cursor;

    overriding
    function Next
      (Object : Directory_Iterator;
       Value  : Cursor) return  Cursor;

and in the body, I added....

    function Entries
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
       return Directory_Iterator_Interfaces.Forward_Iterator'Class is
    begin
       return It : Directory_Iterator do
          It.Done := False;

          Start_Search
            (Search    => It.Search,
             Directory => Directory,
             Pattern   => Pattern,
             Filter    => Filter);

       end return;

    end Entries;

    function First
      (Object : Directory_Iterator) return Cursor
    is
       Result : Cursor
         (Directory_Entry => Object.Directory_Entry'Unchecked_Access);
    begin
       Get_Next_Entry
         (Search          => Object.Self.Reference.Search,
          Directory_Entry => Object.Self.Reference.Directory_Entry);

       return Result;
    end First;

    function Next
      (Object : Directory_Iterator;
       Value  : Cursor) return Cursor
    is
       Result : Cursor
         (Directory_Entry => Object.Directory_Entry'Unchecked_Access);
    begin

       if More_Entries (Search => Object.Self.Reference.Search) then
          Get_Next_Entry
            (Search          => Object.Self.Reference.Search,
             Directory_Entry => Object.Self.Reference.Directory_Entry);
       else
          End_Search (Search => Object.Self.Reference.Search);
          Result.Done := True;
       end if;

       return Result;
    end Next;

****************************************************************

From: Ed Schonberg
Sent: Sunday, October 4, 2015  9:51 AM

> Which would be better?
>  Use "in" instead of "of" in the loop iterators, returning an iterator
>  object rather than an array? Or the other way round?

I think you make a good argument that using an iterator object is preferable to
constructing a temporary array.  This points to a construct that has become very
popular in other languages:  the generator.  Of course we can write generators
in Ada, but there is no single syntax that makes the idiom immediately
available.  Maybe some new aspect for a protected function?  It would be great
to have a simple way to describe a function where successive calls generate a
stream of values from some collection, without building the collection
explicitly.  Language design invited…

****************************************************************

From: Tucker Taft
Sent: Sunday, October 4, 2015  8:34 PM

> I think you make a good argument that using an iterator object is
> preferable to constructing a temporary array.  This points to a
> construct that has become very popular in other languages:  the generator.
> Of course we can write generators in Ada, but there is no single syntax that
> makes the idiom immediately available.

I would think that an iterator object, or an iterable collection object,
provides what you want.  Can you give an example of what a generator would do
that is not accomplishable with the First/Next sequence of iterators?

> ... Maybe some new aspect for a protected function?

I am not sure I understand why a protected function would be involved.

> ... It would be great to have
> a simple way to describe a function where successive calls generate a
> stream of values from some collection, without building the collection
> explicitly.  Language design invited…

I would certainly hope we could use one of the existing syntaxes "for I in XXX
loop" or "for I of YYY loop" to accomplish this.  The interesting question is
what these would expand into for a "generator" vs. an "iterator" (hard for me
since I am unsure of the difference).

****************************************************************

From: Randy Brukardt
Sent: Monday, October 5, 2015  6:08 PM

...
> I would think that an iterator object, or an iterable collection
> object, provides what you want.  Can you give an example of what a
> generator would do that is not accomplishable with the First/Next
> sequence of iterators?

I was equally puzzled by Ed's message. The only thing I can think of is
generating objects without some sort of Cursor (pointer) involved. But that only
works if:
(A) Objects in the stream are not modifiable, as in a functional language.
(You have to have some sort of access to the object in order to modify it within
the loop.)
(B) The order that the objects are to be returned is inherent in the data
structure. (The cursor provides a "next" location; it's hard to imagine many
uses where that isn't necessary.)

And even then, the implementation of the object stream probably will need some
sort of pointer, so using it as a cursor isn't a real issue.

In the vast majority of cases, you're going to need the operations of a forward
iterator, and since there are only two operations to implement, it's hard to
imagine how it could be made much simpler.

****************************************************************

From: Ed Schonberg
Sent: Tuesday, October 6, 2015  8:42 AM

What generators provide is lazy evaluation, when you are searching over a
potentially unbounded collection and you want to generate its values one by one.
Think of a search over prime numbers, or a crossword puzzle solver that will
show you words of 9 letters when the middle one is a Z!  You don't need to
precompute the set, you want to have a construct with local memory that embodies
the computation to go from one element of the set to the next.   In Brad's
example, the set of values exists in the directory so you can use cursors, but
that may not be the case: in GPS for example there is a typing completion
algorithm that generates dynamically suggestions based on keystrokes so far.  It
is not possible to precompute a set of completions in advance,  and what you
need is to encapsulate that set in a generator that will produce successive
candidates on demand.

****************************************************************

From: Tucker Taft
Sent: Tuesday, October 6, 2015  10:03 AM

> ... It is not possible to precompute a set of completions in advance,
> and what you need is to encapsulate that set in a generator that will produce
> successive candidates on demand.

Is there something that requires an iterator to be static?  Couldn't the call on
"Next" compute or retrieve the next value?

****************************************************************

From: Ed Schonberg
Sent: Tuesday, October 6, 2015  10:21 AM

Yes, but either the caller saves the previous value and Next takes it as an
argument, or else Next needs local memory, which is why coroutines come up in
these discussions.  This is why I mentioned protected types earlier: the
simplest realization of that model  in current Ada is a protected object with
one function and some local storage.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, October 6, 2015  11:39 AM

> Yes, but either the caller saves the previous value and Next takes it
> as an argument,

Which of course is exactly the model of Ada's iterators. For this case, the
"cursor" would be whatever is needed to understand the previous value.

> or else Next needs local memory,

The iterator object provides the local state. (Next having two parameters.)

> which is why coroutines come up in these discussions.  This is why I
> mentioned protected types earlier: the simplest realization of that
> model  in current Ada is a protected object with one function and some
> local storage.

Actually, the existing iterators map it pretty well so far as I can tell. I
recall Bob asking about lazy evaluation of a list during the early design, and
the intent was that they could be used that way. Perhaps a bit more complex than
a single function, but not particularly bad.

And of course, if you don't want the for loop syntax, you can just use a
function and some local state. I don't see anything else that the language could
do to help you in that case (I don't think Ada is getting lazy evaluation or
coroutines anytime in the near future!).

****************************************************************

From: Tucker Taft
Sent: Monday, October 5, 2015  7:30 PM

...
> Other than using the Rosen Trick, the implementation was quite easy
> and uses existing library functions defined by the standard.

What happens if you copy the value of a cursor and save it?  It seems the cursor
always points at the "current" directory entry.

> Looking at the meeting minutes, it was suggested that I look at having
> the Entries function return an array of Directory_Entry_Types, rather
> than an iterator object.  I suspect this is partly because noone had
> yet considered using the Implicit_Dereference aspect as I have.
>
> The main difference to the programmer between what was suggested and
> what I implemented would be that the user would need to use "of"
> instead of "in" in the loop iterator specification.
>
> ie.
>
> for Dir_Entry of Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
>    Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;
>
> Not a huge difference. But there are some others. Returning an array
> could be a problem if the directory consisted of many files. A few
> thousand files might cause stack overflow.

Do you mean secondary stack overflow, because these objects would almost
certainly not reside on the "primary" stack?  That seems relatively unlikely.

Alternatively, did you consider creating a function that returns a (conceptual)
directory-entry "container" rather than an actual array, which could be
allocated on the heap, and use a cursor object containing a reference-counted
pointer to the container, and an index into the container?

> The solution I came up with however scales to any number of files
> because the object being returned is just an iterator object, not an
> array. The iterator object only contains a single Directory_Entry_Type
> object component at a time.

True, but then the "cursor" is a bit of a misnomer, as it is really just a level
of indirection.

> Another difference is that I think perhaps it is better to avoid
> "implicit" containers. There is no array object that the programmer
> can see.

I had presumed that the function would be returning a container that was
indexable, so in what sense is that "implicit"?  It is a temporary, that is
true, though presumably it could be built in one place and iterated multiple
times.

> ... I think using iterable_names are good to use when you have an
> explicit container object declaration in the programmers code, but
> otherwise, an iterator_name seems more appropriate when there is no
> such explicit container.
>
> On the other hand, this (and the environment variables package which
> could be handled the same way) would be the first standard containers
> where the new Ada 2012 iterators are designed mainly for "in" loops rather than "of" loops.
>
> I think considering all the above, I am still leaning towards what I
> have, but wanted to get others opinions.
>
> Which would be better?
>    Use "in" instead of "of" in the loop iterators, returning an
> iterator object rather than an array? Or the other way round? ...

I think "in" or "or" is not that big of an issue.  The more important question
might be what are the semantics of saving the (temporary) container, or saving a
cursor?  Does a cursor end up as a dangling reference.  Do all cursor values
point at the same object?

****************************************************************

From: Randy Brukardt
Sent: Monday, October 4, 2015  6:19 PM

...
> The main difference to the programmer between what was suggested and
> what I implemented would be that the user would need to use "of"
> instead of "in" in the loop iterator specification.

Actually, the main difference is that the implementation-defined behavior when a
directory is altered is avoided. (See A.16(110/3).) The array would clearly be a
snapshot of the directory at the time it is taken, so it wouldn't change
afterwards.

Also, as Tucker notes, copying indexes into the array would be harmless.
Copying one of your cursors would most likely leave it being a dangling pointer
after the completion of the iteration. Sounds like more erroneous execution
possibilities (which I'd prefer to avoid).

I could see ways to avoid cursors becoming erroneous if someone copies them, but
they all boil down to making an "implicit container", which you say you don't
want to do.

****************************************************************

From: Brad Moore
Sent: Thursday, October 8, 2015  1:23 AM

>> This allowed me to write:
>>
>> for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
>>     Put_Line(Directories.Simple_Name (Dir_Entry)); end loop;
>>
>
> What happens if you copy the value of a cursor and save it?  It seems
> the cursor always points at the "current" directory entry.
>

I was kind of hoping that the accessibility rules would somehow prevent
squirreling a temporary access value from an inner scope to an outer scope. The
cursor's discriminant is an access to a Directory_Entry_Type component of the
iterator object. I tried this with GNAT however, and it didn't complain. Copying
inside the loop body should be OK since any copy would be expected to designate
the same "current" directory entry. Shouldn't the accessibility checks prevent
such copying to variables declared in a global scope?

declare
   DE : aliased Directories.Directory_Entry_Type;
   Squirrel : Directories.Cursor (DE'Access); begin

    for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
       Put_Line (Directories.Simple_Name (Dir_Entry));
       Squirrel := Directories.Cursor'(Dir_Entry);
    end loop;
end;

****************************************************************

From: Randy Brukardt
Sent: Thursday, October 8, 2015  1:07 PM

...
> Shouldn't the accessibility checks prevent such copying to variables
> declared in a global scope?
>
> declare
>    DE : aliased Directories.Directory_Entry_Type;
>    Squirrel : Directories.Cursor (DE'Access); begin
>
>     for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
>        Put_Line (Directories.Simple_Name (Dir_Entry));
>        Squirrel := Directories.Cursor'(Dir_Entry);
>     end loop;
> end;

The above is fine for accesibility (I think), but it should raise
Constraint_Error because you can't change the discriminant of Squirrel (thus the
discriminant check should fail). Try: (I didn't compile this)

declare
   type Outer is access all Directories.Directory_Entry_Type;
   Squirrel : Outer;
begin
     for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop
        Put_Line (Directories.Simple_Name (Dir_Entry));
        Squirrel := Dir_Entry.D; -- D should be whatever the name of your discriminant is.
     end loop;
end;

...
>I was kind of hoping that the accessibility rules would somehow prevent squirreling
>a temporary access value from an inner scope to an outer scope. The cursor's
>discriminant is an access to a Directory_Entry_Type component of the iterator object.

They're supposed to do that. You shouldn't be able to copy the discriminant to
any named access type that is outside of the loop.

But Tucker's point is that you could squirrel them between iterations, since all
of the iterations belong to a single master. The good news is I can't figure out
how to write such a thing [if you add a block, you're at a different level];
maybe with a Holder container and a subprogram? Or maybe it's impossible and
Tucker is just throwing FUD.

****************************************************************

From: Tucker Taft
Sent: Thursday, October 8, 2015  4:40 PM

There are a couple of subtle issues here.  Not surprising because we are
tip-toeing very near the heart of darkness (3.10.2).  I think we are probably
OK, but convincing myself of that was not easy.  The loop parameter Dir_Entry is
of the Cursor type, and it is *initialized* by a call on First, so no constraint
check.  The subsequent iterations have Dir_Entry being *assigned* from the
result of Next, so here there *would* be a constraint check, presumably, and if
the access discriminant pointed elsewhere, then that should fail the
discriminant check.  So in fact, if you are going to use this approach, then the
access discriminant of the cursor object must *always* point to the same object.
A bit weird!

As far as squirreling away, I agree that the accessibility level of the access
discriminant's type should be the same as that of the loop parameter (per
3.10.2(12.5/3)), so GNAT should disallow the assignment to an access-type
variable outside the loop if you are assigning from the access discriminant
only, because of the required implicit or explicit conversion (per
4.6(24.17/4)); it should fail a discriminant check if you assign from the loop
parameter as a whole.  It should also disallow using the value of the loop
parameter, or the access discriminant thereof, as the initial value in an
initialized allocator for an access type declared outside of the loop, per
4.8(5.3/3, 10.1/3).

So this looks OK, but it shows some issues with GNAT's discriminant checks
(unless you are suppressing them, or you compiled but did not actually run the
code).  It will be interesting to see whether GNAT rejects Randy's example,
where you are assigning from the access discriminant itself.

After all of this, I am somewhat uncomfortable using this trick to provide an
iterator. Alternatively, we should bless this approach and recommend it for
other cases where you want to use the iterator syntax but not actually use the
cursor as an index into anything, but rather just use it as a level of
indirection into the iterator object, which has the current value stored within
it.   Calling it a "cursor" is of course a bit confusing then.
  It is more like a "handle" on the current object of the iteration, which is
  hidden somewhere within the iterator object itself (in this case) or something
  it references (in the container iterator case).

We are relying on the access discriminant to prevent squirreling things away
(thanks to the discriminant check), as well as giving us the implicit
dereference semantics.  I would be somewhat more comfortable if we hid the
access discriminant, and perhaps the whole "cursor" notion completely.

What would happen if we turned this into a container element iterator, by
changing "Entries" to be something that could follow "of" rather than "in"?
That is, "Entries" would return an iterable object that has a Constant_Indexing
aspect.  The Constant_Indexing aspect would return the "current" Directory_Entry
(or a const reference to the current Directory_Entry if we want to avoid copying
a large Directory_Entry), pretty much independent of the cursor value passed in.
The cursor could be effectively a Boolean value, indicating whether or not there
is another directory entry to be processed, so Has_Element could simply return
the Boolean value of the cursor, and the Constant_Indexing attribute could raise
an exception if passed a cursor with value False.

For example:

    type Entry_Presence is new Boolean; -- the "cursor" type
    function Has_Element(EP : Entry_Presence) return Boolean is (Boolean(EP));

    package Dir_Iterators is
      new Ada.Iterator_Interfaces(Entry_Presence, Has_Element);

    type Dir_Entry_Iterator is  --  The iterable container type
      limited new Dir_Iterators.Forward_Iterator
        with Constant_Indexing => Current_Entry,
             Default_Iterator => Entries,
             Iterator_Element => Dir_Entry;

    function Entries  --  Returns an iterable container
      (Dir_Name : String; Pattern : String)
       return Dir_Entry_Iterator;

    function First   --  One of the forward iterator operations
      (Entries: Dir_Entry_Iterator)
       return Entry_Presence;
    function Next    --  One of the forward iterator operations
      (Entries: Dir_Entry_Iterator; Prev : Entry_Presence)
       return Entry_Presence;

    function Current_Entry  --  "indexing" operation
      (Entries: Dir_Entry_Iterator; EP : Entry_Presence)
       return Dir_Entry
      with Pre => Has_Element(EP);
       --  or could return an object that has an access discrim
       --  e.g.:
       --     type Dir_Ent_Ref(DE : not null access constant Dir_Entry) ...


    for Dir_Ent of Entries("/homes/stt", "*.txt") loop
      ...
    end loop;

This would preserve the model of a directory as a vector of directory entries,
while still allowing us to implement the iteration without building up that
vector explicitly.  It also eliminates the visible use of a reference object as
a cursor, and the concerns about an attempt at squirreling away the cursor
causing a run-time constraint error or a potentially confusing accessibility
error.

****************************************************************

From: Tucker Taft
Sent: Thursday, October 8, 2015  5:00 PM

I think I might have gotten myself confused with the "Default_Iterator" aspect.
Stay tuned for another version of the actual source code (I might actually try
to compile it next time ;-).  I think the basic idea should still work, allowing
us to use "of" rather than "in" and avoid some of the accessibility check
issues.  I just need to get my "iterable container" and "default iterator" and
"iterator element" aspects all straightened out...

****************************************************************

From: Tucker Taft
Sent: Thursday, October 8, 2015  5:13 PM

OK, the change is to make the Default_Iterator aspect denote an identity
function, which takes the Dir_Entry_Iterator and returns it, since the
Dir_Entry_Iterator is acting as both our "iterable container type" and as an
"iterator type."  Hence:

    type Dir_Entry_Iterator is  --  The iterable container type
      new Dir_Iterators.Forward_Iterator
        with Constant_Indexing => Current_Entry,
             Default_Iterator => Identity,
             Iterator_Element => Dir_Entry;

    function Identity(Iter: Dir_Entry_Iterator) return Dir_Entry_Iterator is (Iter);

Note that because we wanted to define the body of Identity using a simple
expression function, rather than an aggregate, we had to change
Dir_Entry_Iterator to be a non-limited type.  Everything but the
Default_Iterator and limitedness of Dir_Entry_Iterator remains the same.

Sorry for the confusion.

****************************************************************

From: Brad Moore
Sent: Thursday, October 8, 2015  8:00 PM

...
> As far as squirreling away, I agree that the accessibility level of
> the access discriminant's type should be the same as that of the loop
> parameter (per 3.10.2(12.5/3)), so GNAT should disallow the assignment
> to an access-type variable outside the loop if you are assigning from
> the access discriminant only, because of the required implicit or
> explicit conversion (per 4.6(24.17/4)); it should fail a discriminant
> check if you assign from the loop parameter as a whole.  It should
> also disallow using the value of the loop parameter, or the access
> discriminant thereof, as the initial value in an initialized allocator
> for an access type declared outside of the loop, per 4.8(5.3/3, 10.1/3).
>
> So this looks OK, but it shows some issues with GNAT's discriminant
> checks (unless you are suppressing them, or you compiled but did not
> actually run the code).

Sorry, it was late last night, when I was trying this out.
I compiled but did not run the code.
When I run it today, I see the Constraint_Error being raised, as expected.

****************************************************************

From: Brad Moore
Sent: Thursday, October 8, 2015  11:46 PM

I took your suggestion and ran with it, but found there were some further tweaks
required.

In particular, I found that I had to change Dir_Entry_Iterator back to being a
limited type. This is because I needed to be able to modify an "in" parameter
(in calls such as First, and Next), so I had to use the Rosen trick to do this.
However, this required the type to be limited. As a result of making it limited,
I had to involve two separate objects, the Container object and the Iterator
object. Both are still the same type, (and maybe that's confusing). Because
Identity needs to return a limited type, it cannot just return its "Iter"
parameter. It has to construct a new return object from the "Iter" parameter. I
decided to put the storage for the Directory_Entry in the Iterator object, but
the Current_Entry function receives the Container object, not the Iterator
object, and needs to ultimately get to the Iterator object, so I had to store an
extra access value from the container to access the iterator object. Only the
container object has this access value. The iterator object sets its own value
for this component to null.

All in all, the implementation was a bit tricky to get working, but it does seem
to work. I have it compiling and executing with proper results in GNAT.

The new content added to the public part of the Ada.Directories package is;

    type Entry_Presence is new Boolean; -- the "cursor" type

    function Has_Element
      (EP : Entry_Presence) return Boolean is (Boolean (EP));

    package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces
      (Cursor      => Entry_Presence,
       Has_Element => Has_Element);

     package Dir_Iterators is
       new Ada.Iterator_Interfaces (Entry_Presence, Has_Element);

     type Dir_Entry_Iterator is limited --  The iterable container type
      new Dir_Iterators.Forward_Iterator with private
        with Constant_Indexing => Current_Entry,
             Default_Iterator => Identity,
             Iterator_Element => Directory_Entry_Type;

    overriding
    function First (Iter : Dir_Entry_Iterator) return Entry_Presence;

    overriding
    function Next
      (Iter : Dir_Entry_Iterator;
       Value  : Entry_Presence) return  Entry_Presence;

    function Current_Entry  --  "indexing" operation
      (Entries : Dir_Entry_Iterator; EP : Entry_Presence)
       return Directory_Entry_Type
      with Pre => Has_Element (EP);

    function Identity (Iter : Dir_Entry_Iterator) return Dir_Entry_Iterator;

    function Entries  --  Returns an iterable container
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
        return Dir_Entry_Iterator;

I think I agree with you that this is probably a better approach than the "in"
syntax version.

****************************************************************

From: Brad Moore
Sent: Saturday, October 10, 2015  12:26 PM

I was able to go further and hide the "noise" parts of the iterator mechanism in
the private part of the package. I ended up creating two separate types to
distinguish the container from the iterator. The Container is a type called
"Directory_Listing", and the Iterator type is defined in the private part of the
package. I was able to move the "Identity" function and the "Current_Entry"
function into the private part of the package as well, since the current
freezing rules allowed me to do this.

I think in this case, it a nice feature because those two functions are not
intended to be called by programmers, and are only needed to support the
iterator mechanism. Thus, I was able to reduce the additions needed for to the
public part of the package to just;

    private with Ada.Iterator_Iterfaces;
    ...

    type Directory_Listing is tagged limited private
       with Constant_Indexing => Current_Entry,
            Default_Iterator => Identity,
            Iterator_Element => Directory_Entry_Type;

    function Entries
      (Directory : String;
       Pattern   : String;
       Filter    : Filter_Type := (others => True))
        return Directory_Listing;

****************************************************************

From: Brad Moore
Sent: Saturday, October 10, 2015  5:59 PM

Here is my writeup for the AI12-0009-1 [This is version /02 of the AI - Editor.]

****************************************************************

From: Tucker Taft
Sent: Saturday, October 10, 2015  8:44 PM

...
> I think in this case, it a nice feature because those two functions
> are not intended to be called by programmers, and are only needed to
> support the iterator mechanism. Thus, I was able to reduce the
> additions needed for to the public part of the package to just;
>
>     private with Ada.Iterator_Iterfaces;
>     ...
>
>     type Directory_Listing is tagged limited private
>        with Constant_Indexing => Current_Entry,
>             Default_Iterator => Identity,
>             Iterator_Element => Directory_Entry_Type;

I don't think you can refer to a declaration that occurs in the private part in
an aspect, if the aspect specification is in the visible part.  See
13.1.1(11/3):

   "The usage names in an aspect_definition are not resolved at the point of the
    associated declaration, but rather are resolved at the end of the
    immediately enclosing declaration list."

****************************************************************

From: Randy Brukardt
Sent: Saturday, October 4, 2015  9:53 PM

Yup. I just finished looking that up to write essentially the same message that
you did. But I see you beat me to it, between the time I read Brad's message on
my phone and the time I arrived in the office to answer it (but mainly to
reprogram my handheld GPS for the northeast).

****************************************************************

From: Bob Duff
Sent: Sunday, October 11, 2015  6:11 AM

> I don't think you can refer to a declaration that occurs in the
> private part in an aspect, if the aspect specification is in the visible part.
> See 13.1.1(11/3):

You can in older versions of GNAT.  This bug was fixed fairly recently.

****************************************************************

From: Randy Brukardt
Sent: Monday, October 12, 2015  3:11 PM

There really ought to be an ACATS test for that. It's one of the higher priority
untested things; any volunteers??

****************************************************************

From: Brad Moore
Sent: Sunday, October 11, 2015  12:22 PM

> I don't think you can refer to a declaration that occurs in the
> private part in an aspect, if the aspect specification is in the visible part.
> See 13.1.1(11/3):
>
>    "The usage names in an aspect_definition are not resolved at the
> point of the associated declaration, but rather are resolved at the
> end of the immediately enclosing declaration list."
>

That's somewhat unfortunate as it means having to pull in more declarations from
the private part that are mostly of no concern to the programmer, and having to
deal with unnecessary use cases.

For example for the function,
   function Current_Entry
      (Entries : Directory_Listing; EP : Entry_Presence)
       return Directory_Entry_Type
      with Pre => Has_Element (EP);

we have to now deal with the possibility that a user might call this function
explicitly with EP having a value of True, while the state of the
Directory_Listing container might be in a state where the current directory
entry is not present.

It makes me think that there needs to be some capability to declare that a
subprogram is not explicitly callable by the programmer, and only callable by
the implementation.

I'm thinking of a User_Callable aspect that can be applied to the subprogram as
in;

  function Current_Entry
    (Entries : Directory_Listing; EP : Entry_Presence)
      return Directory_Entry_Type
    with Pre => Has_Element (EP),
         User_Callable => False;

This would hopefully would mean that less semantics and wording for such a
function would be needed in the RM. I think this helps also to separate the real
public interface from the bits and pieces that are not intended to be called by
user.

****************************************************************

From: Brad Moore
Sent: Sunday, October 11, 2015  3:09 PM

Here is an updated version to replace what I submitted yesterday. [This is
version /03 of the AI - Editor.]

This version eliminates the "bug" where I was assuming I could defer the
Contant_Indexing function and Default_Iterator functions to the private part
of the packages.

This version pulls the necessary declarations from the private part back into
the public part. I have tested this implementation for the Ada.Directories
package, and have a working implementation.

I also renamed Identity to Iterate, as Identity is no longer true and Iterate
is more consistent with the naming we use on other existing containers.

For now, disregard my comment in my previous email about providing a
User_Callable aspect. I don't think it is warranted here.

I think we want to expose Iterate and Current_Entry as user callable.
This allows one to write both forms of generalized iterator loops.
(Using "in" or "of" in the loop iterator_specification)

****************************************************************

From: Randy Brukardt
Sent: Monday, October 12, 2015  3:43 PM

> Here is an updated version to replace what I submitted yesterday.
> This version eliminates the "bug" where I was assuming I could defer
> the Contant_Indexing function and Default_Iterator functions to the
> private part of the packages.

I corrected a couple of typos in this as I was posting it. Specifically,

...
> I also renamed Identity to Iterate, as Identity is no longer true and
> Iterate is more consistent with the naming we use on other existing
> containers.

The Default_Iterator aspect was still set to Identity in A.16(36/3). You
certainly meant that to be Iterate as well.

Also, !question should be !problem in an !Amendment class AI.

****************************************************************

From: Jeff Cousins
Sent: Sunday, January 3, 2016  1:59 PM

> Jeff Cousins: AI12-0009-1 (check wording in /03 version)

There seems to have been a generalised replacement of key-words by their bold
lower-case version, this has resulted in some sentences beginning with a
lower-case letter, e.g.:

for generalized loop iteration

if Pattern is the null string

when Start_Search propagates

At the meeting we said that type Entry_Presence should be private not new
Boolean.  Presumably the same for type Variable_Presence too.

type Directory_Listing

The description of this seems a bit short compared with that for type
Environment_Variable_Listing.  Maybe insert a middle sentence "For generalized
loop iteration (see 5.5.2), an object of the type Directory_Listing can be an
iterable container object for the loop that will generate a forward container
element iterator as the loop iterator and a loop cursor designating each
environment variable of the execution environment."  Otherwise it looks a bit
odd referring to a cursor in the descriptions of Entries and Iterate when the
cursor hasn't yet been introduced to the reader.

function Current_Entry

There are rather a lot of "and"s and "or"s in the first sentence, maybe it needs
a few commas.

type Environment_Variable_Listing is tagged limited private

   with Constant_Indexing => Current_Entry,

should be

type Environment_Variable_Listing is tagged limited private

   with Constant_Indexing => Current_Variable,

Personally, I would prefer All_Environment_Variables to All_Variables.

function Current_Variable

Presumably the return type should be Name_Value_Pair_Type not Name_Value_Pair.

type Name_Value_Pair_Type

Spurious } in the description.

****************************************************************

From: Randy Brukardt
Sent: Thursday, January 7, 2016  10:39 PM

Like Jeff, I had an action item to review the wording in AI12-0009-1.
Presumably Brad will use our comments to create his next update. I'm not going
to look too carefully at Jeff's comments; Brad can get to reconcile them (that's
a lot of fun and I'm glad I don't have to do it here). [BTW, when I say "you" in
here, I'm talking about the AI author, that is, Brad.]

As discussed, Entry_Presence should be private. Replace:

   type Entry_Presence is new Boolean;

   function Has_Element
     (EP : in Entry_Presence) return Boolean is (Boolean (EP));

By:

   type Entry_Presence is private;

   function Has_Element
     (EP : in Entry_Presence) return Boolean;

Note that we'll need a definition of what Has_Element does somewhere. BTW, I'd
rename it to Has_Entry, since there is nothing named "Element" around here, just
a bunch of Entries.

Later, you have:

   function Entries
     (Directory : in String;
      Pattern   : in String;
      Filter    : in Filter_Type := (others => True))
       return Directory_Listing;

   For generalized loop iteration (see 5.5.2), Entries returns an iterable
   container object for the loop that will generate a forward container element
   iterator as the loop iterator and a loop cursor designating each available
   directory entry in the return object. ...

I don't understand this sentence. Entries does whatever it does in any context,
so I don't see what "generalized loop iteration" has to do with anything.
Directory_Listing is an iterable container object by definition, and how that
works is defined by the Ada language (Iterate is called, etc.). So this is the
sort of "says nothing" sentence that we're trying to get rid of in the Standard.
I think you want to say something introductory rather than normative to start
with, maybe something like:

   Function Entries returns an object that can be used in generalized loop
   iteration (see 5.5.2) to visit each available directory entry.

And then continue with the details of how that gets done.

I see Jeff suggested going in the other direction. I don't agree with that, but
he might be right that the introductory text is better put on the type
Directory_Listing. That is, something like:

   An object of type Directory_Listing can be used in generalized loop iteration
   (see 5.5.2) to visit each available directory entry for a particular
   directory search.

We certainly don't need to say anything about how that happens; that's obvious
from the aspects declared on the type declaration (at least if you're familiar
with 5.5.1 and 5.5.2, and we can assume that in the RM).

Then the description of Entries should echo that of Start_Search:

   Returns an object that represents the result of a search in the directory
   named by Directory for entries matching Pattern and Filter.

As always, Simplify! (That's a lot easier to say as a reviewer than as an
author. So please don't throw that back at me on my next AI ;-)

---

function Current_Entry:

... or is not involved in generalized loop iteration ...

Huh?? What does this mean? The default state is that there is no entries, so if
it's not iterating, there are no entries. So there doesn't seem to be any need
to mention this (which is good, 'cause it's nonsense - we don't care how the
object is used, just that the sequence of calls makes sense).

---

A.16(125.a/3):

    Implementation Advice: Directories.Start_Search{,} [and ]
    Directories.Search{, and Entries} should raise Name_Error for malformed
    patterns.

The existing routines here give the package name, and the new one ought to as
well - "Directories.Entries". This is the entry in the Implementation Advice
annex, and a reader of that will not have any idea where this comes from, thus
we have to be explicit.

---

As Jeff mentions, "Variable_Presence" should be changed like "Entry_Presence"
was above. And I'd change Has_Element to "Has_Pair" or "Has_Variable" or
something like that.

---

   type Name_Value_Pair_Type is limited private;

   The type Name_Value_Pair_Type represents a single environment variable of
   the execution environment. These items can only be created by generalized
   loop iteration on the return object of the All_Variables function
   in this package}. Information about the item can be obtained from the
   functions declared in this package. A default-initialized object of this
   type is invalid; objects returned by generalized loop iteration on the
   result of Entries are valid.

"These items can only be created by generalized loop iteration..."?? Huh? It
looks to me like you can get some by calling All_Variables and then calling
Current_Variable. As usual, anything that generalized iteration can do can be
done explicitly!

I'd probably focus on the use (which you did in the first sentence); the second
and third sentences are obvious from the specification, so I'd just drop them.

---

Environment_Variable_Listing: Again, simply and point out that it *can* be used
in a generalized loop iteration, but surely there is no requirement to do so.
And repeating the whole business about how those work doesn't add anything not
in the spec. The less English, the better!

My comments on "Entries" apply to "All_Variables" as well.

---

The wording for Current_Variable depends on the previous fact that
Variable_Presence is Boolean rather than private. And again, what the heck does
"involved in generalized loop iteration" mean? What matters is the sequence of
calls to All_Variables, Iterator, Current_Variable, and so on.

---

A.17(25): you've got "involve" on the brain! Just say "have":

It is a bounded error to call Value {if the call does not have a parameter of
type Name_Value_Pair_Type,} if more than one environment variable exists with
the given name; the possible outcomes are that:

---

A.17(28/2):

Again, you're ignoring the possibility of explicitly calling routines to create
and use an object of Environment_Variable_Listing. It's irrelevant how the
object got created or how it is being used, you'll need some abstract way to
describe the state of when an object of that type is in use (between a call to
Iterate and when Has_Pair becomes False).

---

Jeff wrote:

>There seems to have been a generalised replacement of key-words by
>their bold lower-case version, this has resulted in some sentences
>beginning with a lower-case letter, e.g.:
>
>	for generalized loop iteration
>
>	if Pattern is the null string
>
>	when Start_Search propagates

That's the same problem J-P complained about for AI12-0144-1. The autoformatter
for AIs cannot handle the format of typical library subprogram definitions, as
it starts with some code and then is followed by some indented text. *Never*
worry about the formatted output -- *always* look at the raw text version before
complaining about the "pretty" format.

****************************************************************

From: Brad Moore
Sent: Saturday, May 28, 2016  3:02 PM

I have been looking at my homework regarding adding iterators to
Ada.Directories.

In short, I have a new implementation and API that I would like to propose. If
this new API is favoured, I would write up a new version of the AI to match this
API.

In Vermont, Steve commented that the Entry_Presence type should be a private
type. This type represents the cursor for the directory listing, and currently
was implemented as a Boolean type.

I found it improved the abstraction quite a bit to rename this type to be
"Cursor" (as well as make it private), then it fits better with existing
container types, where the Cursor is used to iterate through the directory
listing, rather than a Boolean type.

I then went further, and thought about someone wanting to iterate through the
directory in a particular order, or to have better control over which files are
iterated over.

This led me to a new prototype, where one can optionally specify a sort order
for the results, and a filter that gives much more control than the existing
one, which only lets you specify file type.

I have implemented this prototype in GNAT, and it appears to work. Also, it
seems worth noting that my existing test program did not require any changes to
switch over to this new API.

Internally, the implementation changed from just wrapping calls to Search, to
fetching a vector of matching directory entries, and storing that list in the
Directory_Listing type.

This also allows one to query how many items match the search criteria, via a
Length subprogram. The Directory_Listing type ends up being a finalized type
that looks more like one of the other standard Containers.

A couple of other notes:
I added Preelaborable_Initialization to both the Directory_Listing and the
Cursor type, which wasn't in the previous writeup of the AI.

Also. I think
with Ada.Finalization;

should be changed to

with private Ada.Finalization;

Since all the finalization details are all in the private part.

With these changes, the additions to the Ada.Directories package would look like the following;

    --  Searches in the directory named by Directory for entries matching
    --  Pattern. The subprogram designated by Process is called with each
    --  matching entry in turn. Pattern represents a pattern for
    --  matching file names. If Pattern is null, all items in the
    --  directory are matched;
    --  otherwise, the interpretation of Pattern is implementation-
    --  defined.
    --  Only items that match Filter will be returned. A null Filter
    --  does not apply a filter, otherwise only items in the directory
    --  that return True from the filter, are included in the search.
    --  The ordering of the results is determined by the Sorting
    --  parameter.
    --  By default, the results are sorted in alphanumeric order, but the
    --  Sorting parameter can be specified which returns True if the Left
    --  Item should appear bnefore the Right item in the sort order.
    --  The exception Name_Error is propagated if the string given by
    --  Directory does not identify an existing directory, or if Pattern
    --  does not allow the identification of any possible external file
    --  or directory.
    --  The exception Use_Error is propagated if the external
    --  environment does not support the searching of the directory with
    --  the given name (in the absence of Name_Error).

    type Directory_Listing is tagged limited private
      with Preelaborable_Initialization,
           Constant_Indexing => Current_Entry,
           Default_Iterator => Iterate,
           Iterator_Element => Directory_Entry_Type;

    function Alphanumeric
        (Left, Right : Directory_Entry_Type) return Boolean;

    function Entries  --  Returns an iterable container
      (Directory : String;
       Pattern   : String;
       Filter    : access function (Item : Directory_Entry_Type)
                                      return Boolean := null;
       Sorting   : not null access
                    function (Left, Right : Directory_Entry_Type)
                           return Boolean := Alphanumeric'Access)
        return Directory_Listing;

    function Length
      (Container : Directory_Listing) return Ada.Containers.Count_Type;

    type Cursor is private
      with Preelaborable_Initialization;

    function Has_Element
      (Position : Cursor) return Boolean;

    package Directory_Iterators is
       new Ada.Iterator_Interfaces (Cursor, Has_Element);

    function Current_Entry  --  "indexing" operation
      (Entries : Directory_Listing; Position : Cursor)
       return Directory_Entry_Type
      with Pre => Has_Element (Position);

    function Iterate
      (Listing : Directory_Listing)
       return Directory_Iterators.Forward_Iterator'Class;


Example of Usage:

    for Dir_Entry of Ada.Directories.Entries ("/home/brad", "*.txt") loop
       Put_Line (Ada.Directories.Simple_Name (Dir_Entry)));
    end loop;

I considered whether the Filter and Sort should be generic formals instead of
anonymous subprograms, but I think this would be inconvenient for users to have
to instantiate generics. Also, one might want to process various filters and
sort criteria and it is nicer to be able to do that on the fly in my opinion.

Do these changes look worthwhile to writeup?

****************************************************************

From: Brad Moore
Sent: Saturday, May 28, 2016  10:46 PM

I now have a working implementation of Ada.Environment_Variables as well.

I have a few notes and comments about this design as well that differs from the
previous writeup.

1) Similar to how the pattern parameter of Ada.Directories is useful to limit
   the set of entries involved in the iteration, I added a similar pattern
   parameter to the new iterator for the Ada Environment Variables. In my
   experience, when I am looking for environment variables, I typically am only
   interested in those that match a certain pattern. Eg, "GTK*" could be used to
   find all the environment variables associated with GTK. The default for the
   pattern is "*" which retrieves all variables.

2) I find that when using the new Ada 2012 iterator syntax, I want to use the
   object prefix view notation.

    eg.

    for Variable of Ada.Environment_Variables.All_Variables
      (Pattern => "PARALLEL*") loop
       Put_Line (Variable.Name & "= " & Variable.Value);
    end loop;

rather than

for Variable of Ada.Environment_Variables.All_Variables
(Pattern => "PARALLEL*") loop
    Put_Line (Ada.Environment_Variables.Name (Variable) & "= " &
              Ada.Environment_Variables.Value (Variable)); end loop;

To achieve this, the Name_Value_Pair type needs to be a tagged type.

One issue with having an iterator return an object of a tagged type, is you run
into having to create an indexing function that is dispatching on more than one
type, which doesn't compile.

To get around this, I had to create a nested package for the Name_Value_Pair
type, so that the function tied to the Constant_Indexing aspect of the
Environment Variable listing can be dispatching only on the listing type.

I don't mind that too much, because the name value pair is separated as a
separate abstraction. It might even be worth considering moving that to its own
package. Maybe someone has a better suggestion.

Another benefit of the approach of retrieving the full list and storing in the
listing object is that it removes the need for some of the Rosen Trick
difficulty I ran into, in the previous implementations. It doesn't completely
remove this, as the Rosen Trick is still needed for the Default_Iterator aspect,
because that function has an "in" parameter for the container, but typically the
iterator needs to store a reference to the container.

This issue is actually common to all the standard containers, which is why I
believe GNAT appears to be using 'Unrestricted_Access in all the standard
containers for the Default_Iterator aspects. Using such an attribute would
eliminate the need for using the Rosen Trick and also gets around the issue of
having to make the container a limited type, but unfortunately, this is a
non-portable technique.

Presumably other compiler vendors would need to use a similar implementation
defined attribute, or some other technique to get around this issue.

Ideally, the parameter would have been "in out" which would have avoided these
issues.

Not sure what can be done about that now though...

The Design I have for the Ada.Environment_Variables package adds the following
to the public part of the spec...

Unless I hear back any comments in the next couple of days, I will proceed to
write up the new version of the AI to include this.

    package Name_Value_Pairs is
       type Name_Value_Pair is tagged private;

       ------------------------------------
       -- Operations on Name_Value_Pairs --
       ------------------------------------

       function Create (Name, Value : String) return Name_Value_Pair;
       function Name   (Name_Value  : Name_Value_Pair) return String;
       function Value  (Name_Value  : Name_Value_Pair) return String;

    private
        ...
    end Name_Value_Pairs;

    use Name_Value_Pairs;

    type Environment_Variable_Listing is tagged limited private
       with Preelaborable_Initialization,
            Constant_Indexing => Current_Variable,
            Default_Iterator => Iterate,
            Iterator_Element => Name_Value_Pair;

    function All_Variables
      (Pattern : String := "*")
        return Environment_Variable_Listing;

    function Length
      (Container : Environment_Variable_Listing)
       return Ada.Containers.Count_Type;

    type Cursor is private
      with Preelaborable_Initialization;

    function Has_Element (Position : Cursor) return Boolean;

    package Environment_Iterators is new
      Ada.Iterator_Interfaces (Cursor      => Cursor,
                               Has_Element => Has_Element);

    function Current_Variable  --  "indexing" operation
      (Variables : Environment_Variable_Listing; Position : Cursor)
       return Name_Value_Pair
      with Pre => Has_Element (Position);

    function Iterate
      (Listing : Environment_Variable_Listing)
      return Environment_Iterators.Forward_Iterator'Class;

****************************************************************

From: Randy Brukardt
Sent: Monday, May 30, 2016  5:55 PM

...
> I then went further, and thought about someone wanting to iterate
> through the directory in a particular order, or to have better control
> over which files are iterated over.
>
> This led me to a new prototype, where one can optionally specify a
> sort order for the results, and a filter that gives much more control
> than the existing one, which only lets you specify file type.

I think this is a bad idea. (See below.)

> I have implemented this prototype in GNAT, and it appears to work.
> Also, it seems worth noting that my existing test program did not
> require any changes to switch over to this new API.
>
> Internally, the implementation changed from just wrapping calls to
> Search, to fetching a vector of matching directory entries, and
> storing that list in the Directory_Listing type.

And this is why. This means that the memory requirements for a directory search
are unpredictable, and potentially much, much more than the basic iterators as
currently defined.

In particular, on Windows, the search API uses a handle and some calls to
implement the existing searching capabilities. There is no buffering needed
other than for the value being currently returned. (I believe that Linux would
be similar, but it's been long enough since I've done that I don't want to be
definitive.)

Contrast that to the (dynamic) memory requirements needed to store the entire
directory. And remember that a directory can have nearly an unlimited set of
files; I have directories with nearly 10,000 files (there is one with 9522 files
as I write this). We surely don't want a disincentive to use the "improved"
forms.

****************************************************************

From: Brad Moore
Sent: Tuesday, May 31, 2016  10:32 PM

> I think this is a bad idea. (See below.)
>
>> I have implemented this prototype in GNAT, and it appears to work.
>> Also, it seems worth noting that my existing test program did not
>> require any changes to switch over to this new API.
>>
>> Internally, the implementation changed from just wrapping calls to
>> Search, to fetching a vector of matching directory entries, and
>> storing that list in the Directory_Listing type.
>
> And this is why. This means that the memory requirements for a
> directory search are unpredictable, and potentially much, much more
> than the basic iterators as currently defined.

On linux, there are two functions.

struct dirent *readdir(DIR *dirp);

Which would be similar to the Windows API you describe, and the existing Ada
API.

There is also;

int scandir(const char *dirp, struct dirent ***namelist,
               int (*filter)(const struct dirent *),
               int (*compar)(const struct dirent **,
               const struct dirent **));

Which is where I found the inspiration for the suggestion, and is very similar
to what I proposed.

I could see it being useful to have both these forms available. One for the case
where the directories are large, and you generally want to process all the
files, and the other when storage is not an issue, but a need for processing
files in a certain order, or filtering on certain files, such as those within a
certain date range, or file size.

Only the filenames that satisfy the pattern and filter would actually get stored
in the container.

Maybe it is worth considering having both forms?

For environment variables though, I think it is more reasonable to have the list
stored in the "container", rather than the iterator, mostly because environment
variables are not typically or even likely to be encountered in the numbers that
you might find in a directory. I doubt it makes sense to have two forms for this
package though. I'm guessing that you'd still dislike the storage requirements
for this approach though, as having to copy and then finalize that copy of the
environment may be undesirable, if the OS can in pass the environment, as it
does for C programs, presumably more efficiently as an array of string pointers.

So, nix the idea for environment variables, but what about having two forms for
the directory iteration?

****************************************************************

From: Randy Brukardt
Sent: Tuesday, May 31, 2016  11:13 PM

...
> So, nix the idea for environment variables, but what about having two
> forms for the directory iteration?

That seems like overkill to me. It's easy enough to write it yourself if you
need it (given the existence of the indefinite vectors). The cost for many of my
programs would be prohibitive (remember, one directory has 9522 files). And
you've got a lot of RM text there just for one version.

I'd probably be a bit less negative if we were just talking about a single
procedure, as in:

    procedure Search (
       Directory : in String;
       Pattern   : in String;
       Sort_Order: in Sort_Order_Type; -- Not defaulted for compatibility.
       Filter    : in Filter_Type := (others => True);
       Process   : not null access procedure (
           Directory_Entry : in Directory_Entry_Type));

This would work well with Tucker's (or is it Bob's) lambda loops.

Note that we'll probably consider the lambda loops first, because if we decide
to go that way, we'll probably leave Environment_Variables and Directories
unchanged. (Making all of your work a waste, other than to prove it didn't work
very well -- I guess an important data point that we needed anyway.) My
recollection of the reason that Bob/Tucker came up with that idea was an
alternative way to get the functionality that you were struggling with.

****************************************************************

From: Brad Moore
Sent: Sunday, June 5, 2016  5:42 PM

Here is another part of my homework. [This is version /04 of the AI - Ed.]
This is an update to an existing AI about adding Ada 2012 iterator support to
Ada.Directories and Ada.Environment_Environment_Variables.

I basically addressed Jeff and Randy's comments that arrived subsequent to
our last meeting in Vermont.

There were a few other minor changes as well, such as renaming the
Entry_Presence type to a Cursor type, which I found reads a lot better now.

****************************************************************

From: Randy Brukardt
Sent: Monday, June 6, 2016  7:10 PM

> AI23-009-1

I think I just had a Rip-van-Winkle moment. ;-)

A couple of comments (I'll fix these in the posted draft, so you (Brad) don't
have to):

>  type Directory_Listing is tagged limited private
>     with Preelaborable_Initialization,
>          Constant_Indexing => Current_Entry,
>          Default_Iterator => Iterate,
>          Iterator_Element => Directory_Entry_Type;

Preelaborable_Initialization is not an aspect; it is view-specific which
doesn't map well to aspects. We tried and punted on it for Ada 2012. Perhaps
we should try again to make it an aspect (separate AI, of course), but it
certainly isn't one now. So you have to use the pragma. As in:

  type Directory_Listing is tagged limited private
     with Constant_Indexing => Current_Entry,
          Default_Iterator => Iterate,
          Iterator_Element => Directory_Entry_Type;
  pragma Preelaborable_Initialization (Directory_Listing);

There's some other places like that.

In 112/3:

>The type Directory_Listing need finalization.

This should be "needs finalization".

****************************************************************

From: Brad Moore
Sent: Monday, June 6, 2016  9:53 PM

> A couple of comments (I'll fix these in the posted draft, so you 
> (Brad) don't have to):

Thanks, I also realized I forgot to mention why I made Name_Value_Pair_Type
a tagged type, and thought it would be worth mentioning. Would it be possible
to have you attach the following to replace the last sentence of the
Discussion section, or should I resubmit?

"A similar approach is applied to Ada.Environment_Variables, except that the
type Name_Value_Pair_Type was made to be a tagged type to improve usability by
allowing object prefix notation.

for Pair of Ada.Environment_Variables.Environment loops
    Put_Line (Pair.Name & "=" & Pair.Value); end loop;

rather than;

for Pair of Ada.Environment_Variables.Environment loops
    Put_Line (Ada.Environment_Variables.Name (Pair) & "=" &
              Ada.Environment_Variables.Value (Pair)); end loop;

Making Name_Value_Pair_Type tagged however, meant having to declare the type
in a nested package, to avoid problems with having the call Current_Variable
dispatching on more than one type, which is illegal.

It would have been nice to do the same for Directory_Entry_Type, and make it
a tagged type but that type already existed and changing that to be a tagged
type would likely break backwards incompatibility in some way."

>>   type Directory_Listing is tagged limited private
>>      with Preelaborable_Initialization,
>>           Constant_Indexing => Current_Entry,
>>           Default_Iterator => Iterate,
>>           Iterator_Element => Directory_Entry_Type;
>
> Preelaborable_Initialization is not an aspect; it is view-specific 
> which doesn't map well to aspects. We tried and punted on it for Ada 
> 2012. Perhaps we should try again to make it an aspect (separate AI, 
> of course), but it certainly isn't one now. So you have to use the pragma. As in:
>
>    type Directory_Listing is tagged limited private
>       with Constant_Indexing => Current_Entry,
>            Default_Iterator => Iterate,
>            Iterator_Element => Directory_Entry_Type;
>    pragma Preelaborable_Initialization (Directory_Listing);
>
> There's some other places like that.

I forgot about that, but I note that the code did compile, so apparently GNAT
supports this already. I think it seems warranted in cases like this to have
it as an aspect, if for readability sake if nothing else.

****************************************************************

From: Randy Brukardt
Sent: Monday, June 6, 2016  11:16 PM

> Thanks, I also realized I forgot to mention why I made 
> Name_Value_Pair_Type a tagged type, and thought it would be worth 
> mentioning. Would it be possible to have you attach the following to 
> replace the last sentence of the Discussion section, or should I 
> resubmit?

I can put this wording in, but where does it go? It doesn't seem to fit with
any of the existing discussion.
 
...
> >    type Directory_Listing is tagged limited private
> >       with Constant_Indexing => Current_Entry,
> >            Default_Iterator => Iterate,
> >            Iterator_Element => Directory_Entry_Type;
> >    pragma Preelaborable_Initialization (Directory_Listing);
> >
> > There's some other places like that.
> 
> I forgot about that, but I note that the code did compile, so 
> apparently GNAT supports this already.

Hopefully not in "pedantic" mode. :-)

> I think it seems
> warranted in cases like this to have it as an aspect, if for 
> readability sake if nothing else.

Sure, that seems great. But it's not a (sub)type-related aspect semantically,
as those have the same value for all views. (PI can be false for a partial
view and true for the full view, and both have to work as expected for
compatibility.) Abandoning that means that visibility begins to matter about
(sub)type-related aspects, which means lots of new rules (and likely new bugs).

One can imagine someone adding exceptions all over the Standard for PI (as the
only view-specific type-related aspect), but I'm not volunteering for that.

Or I suppose one could say something like "Notwithstanding what this Standard
says elsewhere, the properties of aspect Preelaborable_Initialization are
exactly as specified here; no other rules for type-related aspects apply." And
then write everything that you need. I ain't signing up to do that, either.

(Steve volunteered to try for Ada 2012, but the full group told him not to
bother, given that it was too close to the final date. Maybe you can talk him
into trying again??)

Anyway, find someone to propose an AI (with wording!!!), and I'll be happy to
tear it to shreds^H^H^H^H^Hput it on the agenda.

In the meantime, use the pragma in any AIs proposed.

****************************************************************

From: Brad Moore
Sent: Tuesday, June 7, 2016  7:52 AM

> I can put this wording in, but where does it go? It doesn't seem to 
> fit with any of the existing discussion.
>

I was thinking at the very end of the discussion, to replace the last sentence.
But perhaps it should start off differently. Something like;

"The overall approach taken for Ada.Environment_Variables is similar to the one
taken for Ada.Directories except that..."

****************************************************************


Questions? Ask the ACAA Technical Agent