!standard A.16(3/2) 15-10-11 AI12-0009-1/03 !standard A.16(36.1/3) !standard A.16(98/2) !standard A.16(112.1/3) !standard A.16(125/3) !standard A.17(3/2) !standard A.17(8/3) !standard A.17(23/3) !standard A.17(25/2) !standard A.17(28/2) !class Amendment 11-11-08 !status work item 11-11-08 !status received 11-10-01 !priority Low !difficulty Medium !subject Iterators for Directories and Environment_Variables !summary Support for generalized loop iteration for Ada.Directories and Ada.Environment_Variables is added. !problem Ada.Directories and Ada.Environment_Variables contain iterator procedures with call-back procedures. There should be a version of the Ada 2012 iterators for these subprograms, so that users can write a loop. !proposal (See wording.) !wording Modify A.16(3/2) "with Ada.IO_Exceptions; with Ada.Calendar; {with Ada.Iterator_Interfaces;} package Ada.Directories is" Append after A.16(36/3) the following -- Directory Iteration type Directory_Listing is tagged limited private with Constant_Indexing => Current_Entry, Default_Iterator => Iterate, Iterator_Element => Directory_Entry_Type; function Entries (Directory : in String; Pattern : in String; Filter : in Filter_Type := (others => True)) return Directory_Listing; type Entry_Presence is new Boolean; function Has_Element (EP : in Entry_Presence) return Boolean is (Boolean (EP)); package Directory_Iterators is new Ada.Iterator_Interfaces (Entry_Presence, Has_Element); function Iterate (Listing : in Directory_Listing) return Directory_Iterators.Forward_Iterator'Class; function Current_Entry (Entries : in Directory_Listing; EP : in Entry_Presence) return Directory_Entry_Type with Pre => Has_Element (EP); Modify A.16(98/2) The type Directory_Entry_Type represents a single item in a directory. These items can only be created {either }by the Get_Next_Entry procedure in this package{, or by generalized loop iteration on the return object of the Entries function in this package}. Information about the item can be obtained from the functions declared in this package. A default-initialized object of this type is invalid; objects returned from Get_Next_Entry {and by generalized loop iteration on the result of Entries} are valid. Append after A.16(112/3) type Directory_Listing is tagged limited private with Constant_Indexing => Current_Entry, Default_Iterator => Iterate, Iterator_Element => Directory_Entry_Type; The type Directory_Listing contains the state of a directory search. A default-initialized Directory_Listing object has no entries available. The type Directory_Listing need finalization. function Entries (Directory : in String; Pattern : in String; Filter : in Filter_Type := (others => True)) return Directory_Listing; For generalized loop iteration (see 5.5.2), Entries returns an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each available directory entry in the return object. Entries starts a search in the directory named by Directory for entries matching Pattern and Filter. Pattern represents a pattern for matching file names. If Pattern is the null string, all items in the directory are matched; otherwise, the interpretation of Pattern is implementation-defined. Only items that match Filter will be returned. After a successful call on Entries, the return object may have entries available, but it may have no entries available if no files or directories match Pattern and Filter. The exception Name_Error is propagated if the string given by Directory does not identify an existing directory, or if Pattern does not allow the identification of any possible external file or directory. The exception Use_Error is propagated if the external environment does not support the searching of the directory with the given name (in the absence of Name_Error). When Start_Search propagates Name_Error or Use_Error, the return object will have no entries available. function Iterate (Listing : Directory_Listing) return Directory_Iterators.Forward_Iterator'Class; Iterate returns a forward iterator object (see 5.5.1) that will generate a value for a loop parameter (see 5.5.2) designating each directory entry in Listing, starting with the first directory entry and moving the cursor as per the Next function. The iterator object needs finalization. function Current_Entry (Entries : Directory_Listing; EP : Entry_Presence) return Directory_Entry_Type with Pre => Has_Element (EP); If Entries is a default-initialized object or if EP is true and the current state of Entries has no entries available or is not involved in generalized loop iteration, then Program_Error is propagated. Otherwise, Current_Entry returns the Directory_Entry associated with the current state of the generalized iterator or container element iterator. It is implementation-defined as to whether the results returned by this subprogram are altered if the contents of the directory are altered during the current iteration of the loop (for example, by another program). The exception Use_Error is propagated if the external environment does not support continued searching of the directory represented by Entries. Modify A.16(125/3) Start_Search{,} [and ]Search{, and Entries} should raise Name_Error if Pattern is malformed, but not if it could represent a file in the directory but does not actually do so. Modify A.16(125.a/3) Implementation Advice: Directories.Start_Search{,} [and ] Directories.Search{, and Entries} should raise Name_Error for malformed patterns. Modify A.17(3/2) "{with Ada.Iterator_Interfaces;} package Ada.Environment_Variables is pragma Preelaborate(Environment_Variables);" Append after A.17(8/3) the following type Name_Value_Pair_Type is limited private; type Environment_Variable_Listing is tagged limited private with Constant_Indexing => Current_Entry, Default_Iterator => Iterate, Iterator_Element => Name_Value_Pair_Type; function All_Variables return Environment_Variable_Listing; type Variable_Presence is new Boolean; function Has_Element (VP : in Variable_Presence) return Boolean is (Boolean (VP)); package Environment_Variable_Iterators is new Ada.Iterator_Interfaces (Variable_Presence, Has_Element); function Iterate (Listing : in Environment_Variable_Listing) return Environment_Variable_Iterators.Forward_Iterator'Class; function Current_Variable (Variables : in Environment_Variable_Listing; VP : in Variable_Presence) return Name_Value_Pair with Pre => Has_Element (VP); -- Operations on Name_Value_Pairs function Name (Name_Value_Pair : in Name_Value_Pair_Type) return String; function Value (Name_Value_Pair : in Name_Value_Pair_Type) return String; Append after A.17(24/2) type Name_Value_Pair_Type is limited private; The type Name_Value_Pair_Type represents a single environment variable of the execution environment. These items can only be created by generalized loop iteration on the return object of the All_Variables function in this package}. Information about the item can be obtained from the functions declared in this package. A default-initialized object of this type is invalid; objects returned by generalized loop iteration on the result of Entries are valid. type Environment_Variable_Listing is tagged limited private with Constant_Indexing => Current_Variable, Default_Iterator => Iterate, Iterator_Element => Name_Value_Pair_Type; If the external execution environment supports environment variables, then for generalized loop iteration (see 5.5.2), an object of the type Environment_Variable_Listing can be an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each environment variable of the execution environment. Otherwise, Program_Error is propagated for generalized loop iteration involving the object. The type Environment_Variable_Listing contains the iteration state of the environment variables of the execution environment. A default-initialized Environment_Variable_Listing object represents all environment variables of the execution environment. The type Environment_Variable_Listing need finalization. function All_Variables return Environment_Variable_Listing; If the external execution environment supports environment variables, then for generalized loop iteration (see 5.5.2), All_Variables returns an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each environment variable of the execution environment. Otherwise, Program_Error is propagated. function Iterate (Variables : Environment_Variable_Listing) return Environment_Variable_Iterators.Forward_Iterator'Class; Iterate returns a forward iterator object (see 5.5.1) that will generate a value for a loop parameter (see 5.5.2) designating each environment variable in Variables, starting with the first environment variable and moving the cursor as per the Next function. The iterator object needs finalization. function Current_Variable (Variables : Environment_Variable_Listing; VP : Variable_Presence) return Name_Value_Pair with Pre => Has_Element (VP); If VP is true and the current state of Variables is not involved in generalized loop iteration, then Program_Error is propagated. Otherwise, Current_Variable returns the Name_Value_Pair associated with the current state of the generalized iterator or container element iterator. function Name (Name_Value_Pair : in Name_Value_Pair_Type) return String; Returns the name of the environment variable represented by Name_Value_Pair. function Value (Name_Value_Pair : in Name_Value_Pair_Type) return String; Returns the value of the environment variable represented by Name_Value_Pair. Modify A.17(25/2) It is a bounded error to call Value {if the call does not involve an object of type Name_Value_Pair_Type,} if more than one environment variable exists with the given name; the possible outcomes are that: Modify A.17(28/2) Making calls to the procedures Set or Clear concurrently with calls to any subprogram of package Environment_Variables, or to any instantiation of Iterate, {or during generalized loop iteration of an Environment_Variable_Listing object} results in erroneous execution. !discussion There were two identified approaches to provide this functionality. One was to have a function that returns an iterator_name that may be used as a generalized iterator. The second approach was to provide a function that returns an object of an iterable container type, that could be used to generate a container element iterator. From the user's perspective, these two approaches would involve slightly different syntax, since generalized iterators use "in" in the iterator_specification while container element iterators use "of". For example, a loop involving the first approach might look like; for Dir_Entry in Ada.Directories.Entries ("/etc", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); end loop; whereas for the second approach the same loop would be written as; for Dir_Entry of Ada.Directories.Entries ("/etc", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); end loop; To implement the first approach, the solution found required the use of a cursor type with an access discriminant and the Implicit_Dereference aspect to designate the corresponding Directory_Entry_Type for the loop. This was needed because Directory_Entry_Type is a limited type, and cannot be returned directly as object of the Ada.Iterator_Interfaces subprograms. To implement this in the Ada.Directories package, the following would need to be added to the public part of the package. with Ada.Iterator_Interfaces; ... type Cursor (Directory_Entry : not null access constant Directory_Entry_Type) is private with Implicit_Dereference => Directory_Entry; function Has_Element (Directory_Entry : Cursor) return Boolean; package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces (Cursor => Cursor, Has_Element => Has_Element); function Entries (Directory : String; Pattern : String; Filter : Filter_Type := (others => True)) return Directory_Iterator_Interfaces.Forward_Iterator'Class; It was generally felt that the use of the Implicit_Dereference aspect here was viewed as being a bit of a trick. It would be ideal if a solution could be found that was less "tricky". Also, having to expose a public type with an access discriminant was seen as being undesirable. The second approach eliminated these concerns, because there is no need to involve access discriminants or the Implicit_Dererence aspect. In addition, the second approach is more flexible because it allows the user to write both forms of loops. The second form of the loop above could be written as; declare Listing : Ada.Directories.Directory_Listing := Ada.Directories.Entries ("/etc", "*.txt"); begin for Dir_Entry in Listing.Iterate loop Put_Line (Directories.Simple_Name (Directories.Current_Entry (Entries => Listing, EP => Dir_Entry))); end loop; end; It was also considered whether the container and the iterator type should be the same type. This seemed more confusing and would have required exposing more of the iterator interface in the public part of the package. It also would have been less clear whether one could restart an iteration on the same container object a second time. It was considered whether reverse iteration through the containers should be supported. Since the existing iteration support for Ada.Directories did not support reverse iteration, it was felt that there would be insufficient need for this capability for generalized loop iteration. A similar approach is applied to Ada.Environment_Variables. !ACATS test ** TBD. !appendix From: Brad Moore Sent: Saturday, October 1, 2011 1:37 PM !topic Ada 2012 Iterators for Directory Searches in Ada.Directories !reference Ada 2012 RM A.16 (32/2, 36/2) !from /Brad Moore 11-10-01/ !keywords Iterators !discussion Ada 2012 introduces new iterator syntax for containers. A directory can be viewed as a container for files. The Ada.Directories package provides the Search related subprograms to iterate through the files in a directory. It seems we ought to be able to use the new iterator syntax to also iterate through the files in a directory. This would be more convenient, and express the iteration in a more natural way, similar to how the new syntax is helpful for iterating through containers. **************************************************************** From: Brad Moore Sent: Saturday, October 1, 2011 2:28 PM !topic Ada 2012 iterators for environment variables !reference Ada 2012 RM A.17 (8/3) !from /Brad Moore 11-10-01/ !keywords Iterators, Environment Variables !discussion Ada 2012 introduces new syntax for iterating through containers. The package Ada.Environment_Variables has a subprogram called Iterate that is used to iterate through the environment variables. It seems we ought to be able to use the new iterator syntax to iterate through environment variables as well. This likely would be more convenient and provide a more natural means of expressing this functionality similar to how the new syntax benefits iterating through containers. **************************************************************** From: Brad Moore Sent: Tuesday, September 29, 2015 9:15 PM I have been working on solutions to this AI, and have encountered a bit of a dilemma. I started looking at Ada.Directories, and found I was able to take the existing GNAT implementation, and get a working prototype by adding just the following to the public part of the package specification. type Cursor (Directory_Entry : not null access constant Directory_Entry_Type) is private with Implicit_Dereference => Directory_Entry; function Has_Element (Directory_Entry : Cursor) return Boolean; package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces (Cursor => Cursor, Has_Element => Has_Element); function Entries (Directory : String; Pattern : String; Filter : Filter_Type := (others => True)) return Directory_Iterator_Interfaces.Forward_Iterator'Class; I found the Implicit_Dereference aspect helpful here, as it allowed me to create a cursor that can designate the limited type Directory_Entry_Type. This allowed me to write: for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop Put_Line(Directories.Simple_Name (Dir_Entry)); end loop; Where I have defined an iterator_name, rather than an iterable_name (see 5.5.2). Other than using the Rosen Trick, the implementation was quite easy and uses existing library functions defined by the standard. Looking at the meeting minutes, it was suggested that I look at having the Entries function return an array of Directory_Entry_Types, rather than an iterator object. I suspect this is partly because noone had yet considered using the Implicit_Dereference aspect as I have. The main difference to the programmer between what was suggested and what I implemented would be that the user would need to use "of" instead of "in" in the loop iterator specification. ie. for Dir_Entry of Ada202x.Directories.Entries ("/home/brad", "*.txt") loop Put_Line(Directories.Simple_Name (Dir_Entry)); end loop; Not a huge difference. But there are some others. Returning an array could be a problem if the directory consisted of many files. A few thousand files might cause stack overflow. The solution I came up with however scales to any number of files because the object being returned is just an iterator object, not an array. The iterator object only contains a single Directory_Entry_Type object component at a time. Another difference is that I think perhaps it is better to avoid "implicit" containers. There is no array object that the programmer can see. I think using iterable_names are good to use when you have an explicit container object declaration in the programmers code, but otherwise, an iterator_name seems more appropriate when there is no such explicit container. On the other hand, this (and the environment variables package which could be handled the same way) would be the first standard containers where the new Ada 2012 iterators are designed mainly for "in" loops rather than "of" loops. I think considering all the above, I am still leaning towards what I have, but wanted to get others opinions. Which would be better? Use "in" instead of "of" in the loop iterators, returning an iterator object rather than an array? Or the other way round? If it is helpful to look at the rest of the implementation I came up with, here is what I added to the private part of the specification: type Cursor (Directory_Entry : not null access constant Directory_Entry_Type) is record Done : Boolean; -- Control : Reference_Control_Type := -- raise Program_Error with "uninitialized reference"; -- The RM says, "The default initialization of an object of -- type Constant_Reference_Type or Reference_Type propagates -- Program_Error." end record; function Has_Element (Directory_Entry : Cursor) return Boolean is (not Directory_Entry.Done); type Directory_Iterator; type Holder (Reference : access Directory_Iterator) is limited null record; type Directory_Iterator is limited new Directory_Iterator_Interfaces.Forward_Iterator with record -- Rosen trick needed so that First and Next can -- modify the iterator object Self : Holder (Directory_Iterator'Access); Search : Search_Type; Directory_Entry : aliased Directory_Entry_Type; end record; overriding function First (Object : Directory_Iterator) return Cursor; overriding function Next (Object : Directory_Iterator; Value : Cursor) return Cursor; and in the body, I added.... function Entries (Directory : String; Pattern : String; Filter : Filter_Type := (others => True)) return Directory_Iterator_Interfaces.Forward_Iterator'Class is begin return It : Directory_Iterator do It.Done := False; Start_Search (Search => It.Search, Directory => Directory, Pattern => Pattern, Filter => Filter); end return; end Entries; function First (Object : Directory_Iterator) return Cursor is Result : Cursor (Directory_Entry => Object.Directory_Entry'Unchecked_Access); begin Get_Next_Entry (Search => Object.Self.Reference.Search, Directory_Entry => Object.Self.Reference.Directory_Entry); return Result; end First; function Next (Object : Directory_Iterator; Value : Cursor) return Cursor is Result : Cursor (Directory_Entry => Object.Directory_Entry'Unchecked_Access); begin if More_Entries (Search => Object.Self.Reference.Search) then Get_Next_Entry (Search => Object.Self.Reference.Search, Directory_Entry => Object.Self.Reference.Directory_Entry); else End_Search (Search => Object.Self.Reference.Search); Result.Done := True; end if; return Result; end Next; **************************************************************** From: Ed Schonberg Sent: Sunday, October 4, 2015 9:51 AM > Which would be better? > Use "in" instead of "of" in the loop iterators, returning an iterator > object rather than an array? Or the other way round? I think you make a good argument that using an iterator object is preferable to constructing a temporary array. This points to a construct that has become very popular in other languages: the generator. Of course we can write generators in Ada, but there is no single syntax that makes the idiom immediately available. Maybe some new aspect for a protected function? It would be great to have a simple way to describe a function where successive calls generate a stream of values from some collection, without building the collection explicitly. Language design invited… **************************************************************** From: Tucker Taft Sent: Sunday, October 4, 2015 8:34 PM > I think you make a good argument that using an iterator object is > preferable to constructing a temporary array. This points to a > construct that has become very popular in other languages: the generator. > Of course we can write generators in Ada, but there is no single syntax that > makes the idiom immediately available. I would think that an iterator object, or an iterable collection object, provides what you want. Can you give an example of what a generator would do that is not accomplishable with the First/Next sequence of iterators? > ... Maybe some new aspect for a protected function? I am not sure I understand why a protected function would be involved. > ... It would be great to have > a simple way to describe a function where successive calls generate a > stream of values from some collection, without building the collection > explicitly. Language design invited… I would certainly hope we could use one of the existing syntaxes "for I in XXX loop" or "for I of YYY loop" to accomplish this. The interesting question is what these would expand into for a "generator" vs. an "iterator" (hard for me since I am unsure of the difference). **************************************************************** From: Randy Brukardt Sent: Monday, October 5, 2015 6:08 PM ... > I would think that an iterator object, or an iterable collection > object, provides what you want. Can you give an example of what a > generator would do that is not accomplishable with the First/Next > sequence of iterators? I was equally puzzled by Ed's message. The only thing I can think of is generating objects without some sort of Cursor (pointer) involved. But that only works if: (A) Objects in the stream are not modifiable, as in a functional language. (You have to have some sort of access to the object in order to modify it within the loop.) (B) The order that the objects are to be returned is inherent in the data structure. (The cursor provides a "next" location; it's hard to imagine many uses where that isn't necessary.) And even then, the implementation of the object stream probably will need some sort of pointer, so using it as a cursor isn't a real issue. In the vast majority of cases, you're going to need the operations of a forward iterator, and since there are only two operations to implement, it's hard to imagine how it could be made much simpler. **************************************************************** From: Ed Schonberg Sent: Tuesday, October 6, 2015 8:42 AM What generators provide is lazy evaluation, when you are searching over a potentially unbounded collection and you want to generate its values one by one. Think of a search over prime numbers, or a crossword puzzle solver that will show you words of 9 letters when the middle one is a Z! You don't need to precompute the set, you want to have a construct with local memory that embodies the computation to go from one element of the set to the next. In Brad's example, the set of values exists in the directory so you can use cursors, but that may not be the case: in GPS for example there is a typing completion algorithm that generates dynamically suggestions based on keystrokes so far. It is not possible to precompute a set of completions in advance, and what you need is to encapsulate that set in a generator that will produce successive candidates on demand. **************************************************************** From: Tucker Taft Sent: Tuesday, October 6, 2015 10:03 AM > ... It is not possible to precompute a set of completions in advance, > and what you need is to encapsulate that set in a generator that will produce > successive candidates on demand. Is there something that requires an iterator to be static? Couldn't the call on "Next" compute or retrieve the next value? **************************************************************** From: Ed Schonberg Sent: Tuesday, October 6, 2015 10:21 AM Yes, but either the caller saves the previous value and Next takes it as an argument, or else Next needs local memory, which is why coroutines come up in these discussions. This is why I mentioned protected types earlier: the simplest realization of that model in current Ada is a protected object with one function and some local storage. **************************************************************** From: Randy Brukardt Sent: Tuesday, October 6, 2015 11:39 AM > Yes, but either the caller saves the previous value and Next takes it > as an argument, Which of course is exactly the model of Ada's iterators. For this case, the "cursor" would be whatever is needed to understand the previous value. > or else Next needs local memory, The iterator object provides the local state. (Next having two parameters.) > which is why coroutines come up in these discussions. This is why I > mentioned protected types earlier: the simplest realization of that > model in current Ada is a protected object with one function and some > local storage. Actually, the existing iterators map it pretty well so far as I can tell. I recall Bob asking about lazy evaluation of a list during the early design, and the intent was that they could be used that way. Perhaps a bit more complex than a single function, but not particularly bad. And of course, if you don't want the for loop syntax, you can just use a function and some local state. I don't see anything else that the language could do to help you in that case (I don't think Ada is getting lazy evaluation or coroutines anytime in the near future!). **************************************************************** From: Tucker Taft Sent: Monday, October 5, 2015 7:30 PM ... > Other than using the Rosen Trick, the implementation was quite easy > and uses existing library functions defined by the standard. What happens if you copy the value of a cursor and save it? It seems the cursor always points at the "current" directory entry. > Looking at the meeting minutes, it was suggested that I look at having > the Entries function return an array of Directory_Entry_Types, rather > than an iterator object. I suspect this is partly because noone had > yet considered using the Implicit_Dereference aspect as I have. > > The main difference to the programmer between what was suggested and > what I implemented would be that the user would need to use "of" > instead of "in" in the loop iterator specification. > > ie. > > for Dir_Entry of Ada202x.Directories.Entries ("/home/brad", "*.txt") loop > Put_Line(Directories.Simple_Name (Dir_Entry)); end loop; > > Not a huge difference. But there are some others. Returning an array > could be a problem if the directory consisted of many files. A few > thousand files might cause stack overflow. Do you mean secondary stack overflow, because these objects would almost certainly not reside on the "primary" stack? That seems relatively unlikely. Alternatively, did you consider creating a function that returns a (conceptual) directory-entry "container" rather than an actual array, which could be allocated on the heap, and use a cursor object containing a reference-counted pointer to the container, and an index into the container? > The solution I came up with however scales to any number of files > because the object being returned is just an iterator object, not an > array. The iterator object only contains a single Directory_Entry_Type > object component at a time. True, but then the "cursor" is a bit of a misnomer, as it is really just a level of indirection. > Another difference is that I think perhaps it is better to avoid > "implicit" containers. There is no array object that the programmer > can see. I had presumed that the function would be returning a container that was indexable, so in what sense is that "implicit"? It is a temporary, that is true, though presumably it could be built in one place and iterated multiple times. > ... I think using iterable_names are good to use when you have an > explicit container object declaration in the programmers code, but > otherwise, an iterator_name seems more appropriate when there is no > such explicit container. > > On the other hand, this (and the environment variables package which > could be handled the same way) would be the first standard containers > where the new Ada 2012 iterators are designed mainly for "in" loops rather than "of" loops. > > I think considering all the above, I am still leaning towards what I > have, but wanted to get others opinions. > > Which would be better? > Use "in" instead of "of" in the loop iterators, returning an > iterator object rather than an array? Or the other way round? ... I think "in" or "or" is not that big of an issue. The more important question might be what are the semantics of saving the (temporary) container, or saving a cursor? Does a cursor end up as a dangling reference. Do all cursor values point at the same object? **************************************************************** From: Randy Brukardt Sent: Monday, October 4, 2015 6:19 PM ... > The main difference to the programmer between what was suggested and > what I implemented would be that the user would need to use "of" > instead of "in" in the loop iterator specification. Actually, the main difference is that the implementation-defined behavior when a directory is altered is avoided. (See A.16(110/3).) The array would clearly be a snapshot of the directory at the time it is taken, so it wouldn't change afterwards. Also, as Tucker notes, copying indexes into the array would be harmless. Copying one of your cursors would most likely leave it being a dangling pointer after the completion of the iteration. Sounds like more erroneous execution possibilities (which I'd prefer to avoid). I could see ways to avoid cursors becoming erroneous if someone copies them, but they all boil down to making an "implicit container", which you say you don't want to do. **************************************************************** From: Brad Moore Sent: Thursday, October 8, 2015 1:23 AM >> This allowed me to write: >> >> for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop >> Put_Line(Directories.Simple_Name (Dir_Entry)); end loop; >> > > What happens if you copy the value of a cursor and save it? It seems > the cursor always points at the "current" directory entry. > I was kind of hoping that the accessibility rules would somehow prevent squirreling a temporary access value from an inner scope to an outer scope. The cursor's discriminant is an access to a Directory_Entry_Type component of the iterator object. I tried this with GNAT however, and it didn't complain. Copying inside the loop body should be OK since any copy would be expected to designate the same "current" directory entry. Shouldn't the accessibility checks prevent such copying to variables declared in a global scope? declare DE : aliased Directories.Directory_Entry_Type; Squirrel : Directories.Cursor (DE'Access); begin for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); Squirrel := Directories.Cursor'(Dir_Entry); end loop; end; **************************************************************** From: Randy Brukardt Sent: Thursday, October 8, 2015 1:07 PM ... > Shouldn't the accessibility checks prevent such copying to variables > declared in a global scope? > > declare > DE : aliased Directories.Directory_Entry_Type; > Squirrel : Directories.Cursor (DE'Access); begin > > for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop > Put_Line (Directories.Simple_Name (Dir_Entry)); > Squirrel := Directories.Cursor'(Dir_Entry); > end loop; > end; The above is fine for accesibility (I think), but it should raise Constraint_Error because you can't change the discriminant of Squirrel (thus the discriminant check should fail). Try: (I didn't compile this) declare type Outer is access all Directories.Directory_Entry_Type; Squirrel : Outer; begin for Dir_Entry in Ada202x.Directories.Entries ("/home/brad", "*.txt") loop Put_Line (Directories.Simple_Name (Dir_Entry)); Squirrel := Dir_Entry.D; -- D should be whatever the name of your discriminant is. end loop; end; ... >I was kind of hoping that the accessibility rules would somehow prevent squirreling >a temporary access value from an inner scope to an outer scope. The cursor's >discriminant is an access to a Directory_Entry_Type component of the iterator object. They're supposed to do that. You shouldn't be able to copy the discriminant to any named access type that is outside of the loop. But Tucker's point is that you could squirrel them between iterations, since all of the iterations belong to a single master. The good news is I can't figure out how to write such a thing [if you add a block, you're at a different level]; maybe with a Holder container and a subprogram? Or maybe it's impossible and Tucker is just throwing FUD. **************************************************************** From: Tucker Taft Sent: Thursday, October 8, 2015 4:40 PM There are a couple of subtle issues here. Not surprising because we are tip-toeing very near the heart of darkness (3.10.2). I think we are probably OK, but convincing myself of that was not easy. The loop parameter Dir_Entry is of the Cursor type, and it is *initialized* by a call on First, so no constraint check. The subsequent iterations have Dir_Entry being *assigned* from the result of Next, so here there *would* be a constraint check, presumably, and if the access discriminant pointed elsewhere, then that should fail the discriminant check. So in fact, if you are going to use this approach, then the access discriminant of the cursor object must *always* point to the same object. A bit weird! As far as squirreling away, I agree that the accessibility level of the access discriminant's type should be the same as that of the loop parameter (per 3.10.2(12.5/3)), so GNAT should disallow the assignment to an access-type variable outside the loop if you are assigning from the access discriminant only, because of the required implicit or explicit conversion (per 4.6(24.17/4)); it should fail a discriminant check if you assign from the loop parameter as a whole. It should also disallow using the value of the loop parameter, or the access discriminant thereof, as the initial value in an initialized allocator for an access type declared outside of the loop, per 4.8(5.3/3, 10.1/3). So this looks OK, but it shows some issues with GNAT's discriminant checks (unless you are suppressing them, or you compiled but did not actually run the code). It will be interesting to see whether GNAT rejects Randy's example, where you are assigning from the access discriminant itself. After all of this, I am somewhat uncomfortable using this trick to provide an iterator. Alternatively, we should bless this approach and recommend it for other cases where you want to use the iterator syntax but not actually use the cursor as an index into anything, but rather just use it as a level of indirection into the iterator object, which has the current value stored within it. Calling it a "cursor" is of course a bit confusing then. It is more like a "handle" on the current object of the iteration, which is hidden somewhere within the iterator object itself (in this case) or something it references (in the container iterator case). We are relying on the access discriminant to prevent squirreling things away (thanks to the discriminant check), as well as giving us the implicit dereference semantics. I would be somewhat more comfortable if we hid the access discriminant, and perhaps the whole "cursor" notion completely. What would happen if we turned this into a container element iterator, by changing "Entries" to be something that could follow "of" rather than "in"? That is, "Entries" would return an iterable object that has a Constant_Indexing aspect. The Constant_Indexing aspect would return the "current" Directory_Entry (or a const reference to the current Directory_Entry if we want to avoid copying a large Directory_Entry), pretty much independent of the cursor value passed in. The cursor could be effectively a Boolean value, indicating whether or not there is another directory entry to be processed, so Has_Element could simply return the Boolean value of the cursor, and the Constant_Indexing attribute could raise an exception if passed a cursor with value False. For example: type Entry_Presence is new Boolean; -- the "cursor" type function Has_Element(EP : Entry_Presence) return Boolean is (Boolean(EP)); package Dir_Iterators is new Ada.Iterator_Interfaces(Entry_Presence, Has_Element); type Dir_Entry_Iterator is -- The iterable container type limited new Dir_Iterators.Forward_Iterator with Constant_Indexing => Current_Entry, Default_Iterator => Entries, Iterator_Element => Dir_Entry; function Entries -- Returns an iterable container (Dir_Name : String; Pattern : String) return Dir_Entry_Iterator; function First -- One of the forward iterator operations (Entries: Dir_Entry_Iterator) return Entry_Presence; function Next -- One of the forward iterator operations (Entries: Dir_Entry_Iterator; Prev : Entry_Presence) return Entry_Presence; function Current_Entry -- "indexing" operation (Entries: Dir_Entry_Iterator; EP : Entry_Presence) return Dir_Entry with Pre => Has_Element(EP); -- or could return an object that has an access discrim -- e.g.: -- type Dir_Ent_Ref(DE : not null access constant Dir_Entry) ... for Dir_Ent of Entries("/homes/stt", "*.txt") loop ... end loop; This would preserve the model of a directory as a vector of directory entries, while still allowing us to implement the iteration without building up that vector explicitly. It also eliminates the visible use of a reference object as a cursor, and the concerns about an attempt at squirreling away the cursor causing a run-time constraint error or a potentially confusing accessibility error. **************************************************************** From: Tucker Taft Sent: Thursday, October 8, 2015 5:00 PM I think I might have gotten myself confused with the "Default_Iterator" aspect. Stay tuned for another version of the actual source code (I might actually try to compile it next time ;-). I think the basic idea should still work, allowing us to use "of" rather than "in" and avoid some of the accessibility check issues. I just need to get my "iterable container" and "default iterator" and "iterator element" aspects all straightened out... **************************************************************** From: Tucker Taft Sent: Thursday, October 8, 2015 5:13 PM OK, the change is to make the Default_Iterator aspect denote an identity function, which takes the Dir_Entry_Iterator and returns it, since the Dir_Entry_Iterator is acting as both our "iterable container type" and as an "iterator type." Hence: type Dir_Entry_Iterator is -- The iterable container type new Dir_Iterators.Forward_Iterator with Constant_Indexing => Current_Entry, Default_Iterator => Identity, Iterator_Element => Dir_Entry; function Identity(Iter: Dir_Entry_Iterator) return Dir_Entry_Iterator is (Iter); Note that because we wanted to define the body of Identity using a simple expression function, rather than an aggregate, we had to change Dir_Entry_Iterator to be a non-limited type. Everything but the Default_Iterator and limitedness of Dir_Entry_Iterator remains the same. Sorry for the confusion. **************************************************************** From: Brad Moore Sent: Thursday, October 8, 2015 8:00 PM ... > As far as squirreling away, I agree that the accessibility level of > the access discriminant's type should be the same as that of the loop > parameter (per 3.10.2(12.5/3)), so GNAT should disallow the assignment > to an access-type variable outside the loop if you are assigning from > the access discriminant only, because of the required implicit or > explicit conversion (per 4.6(24.17/4)); it should fail a discriminant > check if you assign from the loop parameter as a whole. It should > also disallow using the value of the loop parameter, or the access > discriminant thereof, as the initial value in an initialized allocator > for an access type declared outside of the loop, per 4.8(5.3/3, 10.1/3). > > So this looks OK, but it shows some issues with GNAT's discriminant > checks (unless you are suppressing them, or you compiled but did not > actually run the code). Sorry, it was late last night, when I was trying this out. I compiled but did not run the code. When I run it today, I see the Constraint_Error being raised, as expected. **************************************************************** From: Brad Moore Sent: Thursday, October 8, 2015 11:46 PM I took your suggestion and ran with it, but found there were some further tweaks required. In particular, I found that I had to change Dir_Entry_Iterator back to being a limited type. This is because I needed to be able to modify an "in" parameter (in calls such as First, and Next), so I had to use the Rosen trick to do this. However, this required the type to be limited. As a result of making it limited, I had to involve two separate objects, the Container object and the Iterator object. Both are still the same type, (and maybe that's confusing). Because Identity needs to return a limited type, it cannot just return its "Iter" parameter. It has to construct a new return object from the "Iter" parameter. I decided to put the storage for the Directory_Entry in the Iterator object, but the Current_Entry function receives the Container object, not the Iterator object, and needs to ultimately get to the Iterator object, so I had to store an extra access value from the container to access the iterator object. Only the container object has this access value. The iterator object sets its own value for this component to null. All in all, the implementation was a bit tricky to get working, but it does seem to work. I have it compiling and executing with proper results in GNAT. The new content added to the public part of the Ada.Directories package is; type Entry_Presence is new Boolean; -- the "cursor" type function Has_Element (EP : Entry_Presence) return Boolean is (Boolean (EP)); package Directory_Iterator_Interfaces is new Ada.Iterator_Interfaces (Cursor => Entry_Presence, Has_Element => Has_Element); package Dir_Iterators is new Ada.Iterator_Interfaces (Entry_Presence, Has_Element); type Dir_Entry_Iterator is limited -- The iterable container type new Dir_Iterators.Forward_Iterator with private with Constant_Indexing => Current_Entry, Default_Iterator => Identity, Iterator_Element => Directory_Entry_Type; overriding function First (Iter : Dir_Entry_Iterator) return Entry_Presence; overriding function Next (Iter : Dir_Entry_Iterator; Value : Entry_Presence) return Entry_Presence; function Current_Entry -- "indexing" operation (Entries : Dir_Entry_Iterator; EP : Entry_Presence) return Directory_Entry_Type with Pre => Has_Element (EP); function Identity (Iter : Dir_Entry_Iterator) return Dir_Entry_Iterator; function Entries -- Returns an iterable container (Directory : String; Pattern : String; Filter : Filter_Type := (others => True)) return Dir_Entry_Iterator; I think I agree with you that this is probably a better approach than the "in" syntax version. **************************************************************** From: Brad Moore Sent: Saturday, October 10, 2015 12:26 PM I was able to go further and hide the "noise" parts of the iterator mechanism in the private part of the package. I ended up creating two separate types to distinguish the container from the iterator. The Container is a type called "Directory_Listing", and the Iterator type is defined in the private part of the package. I was able to move the "Identity" function and the "Current_Entry" function into the private part of the package as well, since the current freezing rules allowed me to do this. I think in this case, it a nice feature because those two functions are not intended to be called by programmers, and are only needed to support the iterator mechanism. Thus, I was able to reduce the additions needed for to the public part of the package to just; private with Ada.Iterator_Iterfaces; ... type Directory_Listing is tagged limited private with Constant_Indexing => Current_Entry, Default_Iterator => Identity, Iterator_Element => Directory_Entry_Type; function Entries (Directory : String; Pattern : String; Filter : Filter_Type := (others => True)) return Directory_Listing; **************************************************************** From: Brad Moore Sent: Saturday, October 10, 2015 5:59 PM Here is my writeup for the AI12-0009-1 [This is version /02 of the AI - Editor.] **************************************************************** From: Tucker Taft Sent: Saturday, October 10, 2015 8:44 PM ... > I think in this case, it a nice feature because those two functions > are not intended to be called by programmers, and are only needed to > support the iterator mechanism. Thus, I was able to reduce the > additions needed for to the public part of the package to just; > > private with Ada.Iterator_Iterfaces; > ... > > type Directory_Listing is tagged limited private > with Constant_Indexing => Current_Entry, > Default_Iterator => Identity, > Iterator_Element => Directory_Entry_Type; I don't think you can refer to a declaration that occurs in the private part in an aspect, if the aspect specification is in the visible part. See 13.1.1(11/3): "The usage names in an aspect_definition are not resolved at the point of the associated declaration, but rather are resolved at the end of the immediately enclosing declaration list." **************************************************************** From: Randy Brukardt Sent: Saturday, October 4, 2015 9:53 PM Yup. I just finished looking that up to write essentially the same message that you did. But I see you beat me to it, between the time I read Brad's message on my phone and the time I arrived in the office to answer it (but mainly to reprogram my handheld GPS for the northeast). **************************************************************** From: Bob Duff Sent: Sunday, October 11, 2015 6:11 AM > I don't think you can refer to a declaration that occurs in the > private part in an aspect, if the aspect specification is in the visible part. > See 13.1.1(11/3): You can in older versions of GNAT. This bug was fixed fairly recently. **************************************************************** From: Randy Brukardt Sent: Monday, October 12, 2015 3:11 PM There really ought to be an ACATS test for that. It's one of the higher priority untested things; any volunteers?? **************************************************************** From: Brad Moore Sent: Sunday, October 11, 2015 12:22 PM > I don't think you can refer to a declaration that occurs in the > private part in an aspect, if the aspect specification is in the visible part. > See 13.1.1(11/3): > > "The usage names in an aspect_definition are not resolved at the > point of the associated declaration, but rather are resolved at the > end of the immediately enclosing declaration list." > That's somewhat unfortunate as it means having to pull in more declarations from the private part that are mostly of no concern to the programmer, and having to deal with unnecessary use cases. For example for the function, function Current_Entry (Entries : Directory_Listing; EP : Entry_Presence) return Directory_Entry_Type with Pre => Has_Element (EP); we have to now deal with the possibility that a user might call this function explicitly with EP having a value of True, while the state of the Directory_Listing container might be in a state where the current directory entry is not present. It makes me think that there needs to be some capability to declare that a subprogram is not explicitly callable by the programmer, and only callable by the implementation. I'm thinking of a User_Callable aspect that can be applied to the subprogram as in; function Current_Entry (Entries : Directory_Listing; EP : Entry_Presence) return Directory_Entry_Type with Pre => Has_Element (EP), User_Callable => False; This would hopefully would mean that less semantics and wording for such a function would be needed in the RM. I think this helps also to separate the real public interface from the bits and pieces that are not intended to be called by user. **************************************************************** From: Brad Moore Sent: Sunday, October 11, 2015 3:09 PM Here is an updated version to replace what I submitted yesterday. [This is version /03 of the AI - Editor.] This version eliminates the "bug" where I was assuming I could defer the Contant_Indexing function and Default_Iterator functions to the private part of the packages. This version pulls the necessary declarations from the private part back into the public part. I have tested this implementation for the Ada.Directories package, and have a working implementation. I also renamed Identity to Iterate, as Identity is no longer true and Iterate is more consistent with the naming we use on other existing containers. For now, disregard my comment in my previous email about providing a User_Callable aspect. I don't think it is warranted here. I think we want to expose Iterate and Current_Entry as user callable. This allows one to write both forms of generalized iterator loops. (Using "in" or "of" in the loop iterator_specification) **************************************************************** From: Randy Brukardt Sent: Monday, October 12, 2015 3:43 PM > Here is an updated version to replace what I submitted yesterday. > This version eliminates the "bug" where I was assuming I could defer > the Contant_Indexing function and Default_Iterator functions to the > private part of the packages. I corrected a couple of typos in this as I was posting it. Specifically, ... > I also renamed Identity to Iterate, as Identity is no longer true and > Iterate is more consistent with the naming we use on other existing > containers. The Default_Iterator aspect was still set to Identity in A.16(36/3). You certainly meant that to be Iterate as well. Also, !question should be !problem in an !Amendment class AI. **************************************************************** From: Jeff Cousins Sent: Sunday, January 3, 2016 1:59 PM > Jeff Cousins: AI12-0009-1 (check wording in /03 version) There seems to have been a generalised replacement of key-words by their bold lower-case version, this has resulted in some sentences beginning with a lower-case letter, e.g.: for generalized loop iteration if Pattern is the null string when Start_Search propagates At the meeting we said that type Entry_Presence should be private not new Boolean. Presumably the same for type Variable_Presence too. type Directory_Listing The description of this seems a bit short compared with that for type Environment_Variable_Listing. Maybe insert a middle sentence "For generalized loop iteration (see 5.5.2), an object of the type Directory_Listing can be an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each environment variable of the execution environment." Otherwise it looks a bit odd referring to a cursor in the descriptions of Entries and Iterate when the cursor hasn't yet been introduced to the reader. function Current_Entry There are rather a lot of "and"s and "or"s in the first sentence, maybe it needs a few commas. type Environment_Variable_Listing is tagged limited private with Constant_Indexing => Current_Entry, should be type Environment_Variable_Listing is tagged limited private with Constant_Indexing => Current_Variable, Personally, I would prefer All_Environment_Variables to All_Variables. function Current_Variable Presumably the return type should be Name_Value_Pair_Type not Name_Value_Pair. type Name_Value_Pair_Type Spurious } in the description. **************************************************************** From: Randy Brukardt Sent: Thursday, January 7, 2016 10:39 PM Like Jeff, I had an action item to review the wording in AI12-0009-1. Presumably Brad will use our comments to create his next update. I'm not going to look too carefully at Jeff's comments; Brad can get to reconcile them (that's a lot of fun and I'm glad I don't have to do it here). [BTW, when I say "you" in here, I'm talking about the AI author, that is, Brad.] As discussed, Entry_Presence should be private. Replace: type Entry_Presence is new Boolean; function Has_Element (EP : in Entry_Presence) return Boolean is (Boolean (EP)); By: type Entry_Presence is private; function Has_Element (EP : in Entry_Presence) return Boolean; Note that we'll need a definition of what Has_Element does somewhere. BTW, I'd rename it to Has_Entry, since there is nothing named "Element" around here, just a bunch of Entries. Later, you have: function Entries (Directory : in String; Pattern : in String; Filter : in Filter_Type := (others => True)) return Directory_Listing; For generalized loop iteration (see 5.5.2), Entries returns an iterable container object for the loop that will generate a forward container element iterator as the loop iterator and a loop cursor designating each available directory entry in the return object. ... I don't understand this sentence. Entries does whatever it does in any context, so I don't see what "generalized loop iteration" has to do with anything. Directory_Listing is an iterable container object by definition, and how that works is defined by the Ada language (Iterate is called, etc.). So this is the sort of "says nothing" sentence that we're trying to get rid of in the Standard. I think you want to say something introductory rather than normative to start with, maybe something like: Function Entries returns an object that can be used in generalized loop iteration (see 5.5.2) to visit each available directory entry. And then continue with the details of how that gets done. I see Jeff suggested going in the other direction. I don't agree with that, but he might be right that the introductory text is better put on the type Directory_Listing. That is, something like: An object of type Directory_Listing can be used in generalized loop iteration (see 5.5.2) to visit each available directory entry for a particular directory search. We certainly don't need to say anything about how that happens; that's obvious from the aspects declared on the type declaration (at least if you're familiar with 5.5.1 and 5.5.2, and we can assume that in the RM). Then the description of Entries should echo that of Start_Search: Returns an object that represents the result of a search in the directory named by Directory for entries matching Pattern and Filter. As always, Simplify! (That's a lot easier to say as a reviewer than as an author. So please don't throw that back at me on my next AI ;-) --- function Current_Entry: ... or is not involved in generalized loop iteration ... Huh?? What does this mean? The default state is that there is no entries, so if it's not iterating, there are no entries. So there doesn't seem to be any need to mention this (which is good, 'cause it's nonsense - we don't care how the object is used, just that the sequence of calls makes sense). --- A.16(125.a/3): Implementation Advice: Directories.Start_Search{,} [and ] Directories.Search{, and Entries} should raise Name_Error for malformed patterns. The existing routines here give the package name, and the new one ought to as well - "Directories.Entries". This is the entry in the Implementation Advice annex, and a reader of that will not have any idea where this comes from, thus we have to be explicit. --- As Jeff mentions, "Variable_Presence" should be changed like "Entry_Presence" was above. And I'd change Has_Element to "Has_Pair" or "Has_Variable" or something like that. --- type Name_Value_Pair_Type is limited private; The type Name_Value_Pair_Type represents a single environment variable of the execution environment. These items can only be created by generalized loop iteration on the return object of the All_Variables function in this package}. Information about the item can be obtained from the functions declared in this package. A default-initialized object of this type is invalid; objects returned by generalized loop iteration on the result of Entries are valid. "These items can only be created by generalized loop iteration..."?? Huh? It looks to me like you can get some by calling All_Variables and then calling Current_Variable. As usual, anything that generalized iteration can do can be done explicitly! I'd probably focus on the use (which you did in the first sentence); the second and third sentences are obvious from the specification, so I'd just drop them. --- Environment_Variable_Listing: Again, simply and point out that it *can* be used in a generalized loop iteration, but surely there is no requirement to do so. And repeating the whole business about how those work doesn't add anything not in the spec. The less English, the better! My comments on "Entries" apply to "All_Variables" as well. --- The wording for Current_Variable depends on the previous fact that Variable_Presence is Boolean rather than private. And again, what the heck does "involved in generalized loop iteration" mean? What matters is the sequence of calls to All_Variables, Iterator, Current_Variable, and so on. --- A.17(25): you've got "involve" on the brain! Just say "have": It is a bounded error to call Value {if the call does not have a parameter of type Name_Value_Pair_Type,} if more than one environment variable exists with the given name; the possible outcomes are that: --- A.17(28/2): Again, you're ignoring the possibility of explicitly calling routines to create and use an object of Environment_Variable_Listing. It's irrelevant how the object got created or how it is being used, you'll need some abstract way to describe the state of when an object of that type is in use (between a call to Iterate and when Has_Pair becomes False). --- Jeff wrote: >There seems to have been a generalised replacement of key-words by >their bold lower-case version, this has resulted in some sentences >beginning with a lower-case letter, e.g.: > > for generalized loop iteration > > if Pattern is the null string > > when Start_Search propagates That's the same problem J-P complained about for AI12-0144-1. The autoformatter for AIs cannot handle the format of typical library subprogram definitions, as it starts with some code and then is followed by some indented text. *Never* worry about the formatted output -- *always* look at the raw text version before complaining about the "pretty" format. ****************************************************************