Version 1.5 of ais/ai-00283.txt

Unformatted version of ais/ai-00283.txt version 1.5
Other versions for file ais/ai-00283.txt

!standard A.12.01 (28)          01-12-26 AI95-00283/00
!standard A.08.02 (10)
!standard A.08.02 (16)
!class amendment 01-12-27
!status received 01-12-14
!priority Low
!difficulty Medium
!subject Truncation of stream files by Close and Reset
!summary
(TBD.)
!problem
!proposal
!discussion
!example
!ACATS test
!appendix

!topic Inout_File mode missing for Ada.Streams.Stream_Io
!reference RM95-A.12.1(6)
!from Bernard Maudry
!discussion

Would it be possible to add the "Inout_File" mode to
Ada.Streams.Stream_Io.File_Type as Reset truncates a file after the last write
position and it is painful (and sometimes erroneous) to do a dummy Write at the
end of the file before a Reset on this file from Out_File mode to In_File mode.

*************************************************************

From: Randy Brukardt
Date: Friday, December 14, 2001  5:08 PM

This seems like a clear bug in the standard (as modified by the
Corrigendum). We certainly don't need to add a new mode to fix it.

In the discussions for the Corrigendum, we decided that positionable
Stream_IO files acted essentially as direct files. Only non-positionable
files act like sequential files. We thought that we'd fixed the wording to
make that happen, but apparently we missed something.

To actually implement that rule (A.8.2(16), the sentence starting with "In
addition,", which is included by A.12.1(28)) as written, an implementation
would have to keep track of the highest position written since the last
Create/Open/Reset. For a non-positionable file, that is the same as the
current position, but for a positionable file, it could be anywhere. I don't
think it was intended that there was a requirement to save/check the write
location after every write.

So, I think that that sentence in A.8.2(16) is intended to apply only to
non-positionable stream files. This is the only rule in A.8.2 that applies
only to sequential files, so I don't think that there are any more examples
of this particular problem lurking.

There may be good arguments for adding an Inout_File mode to Stream_IO, but
I don't think this is one.

*************************************************************

From: Randy Brukardt
Date: Friday, December 14, 2001  5:15 PM

Oops, of course Close also has this behavior.

And that also is a problem. If you are representing a database index as a
stream file, if you insert a record (and write a couple of items in the
middle), you certainly don't want to have to write at the end so that the
silly system doesn't truncate the file.

Perhaps we need a Truncate operation in Stream_IO for the times when you do
want to truncate?

*************************************************************

From: Pascal Leroy
Date: Thursday, December 20, 2001  4:50 AM

This issue looks like a red herring to me.

If you want to change the mode without truncating, use Set_Mode, not Reset.  It
seems that the only difference between the two is that Set_Mode does not
truncate, but Reset does.

If you want to close the file without truncating, use Set_Mode to change the
mode to In_File before closing.  OK, this means that you need to perform two
operations (Set_Mode + Close).  But the alternative model suggested by Randy
(where Close doesn't truncate and we have an extra Truncate operation) also
requires two operations in the case where you want to truncate-and-close.

Actually, I find the current model (as stated by RM95+TC1) rather natural and
certainly consistent.  But evidently it should be tested by the ACATS since
it's an area where differences between implementations could cause lots of
trouble.

*************************************************************

From: Randy Brukardt
Date: Friday, December 28, 2001  5:23 PM

Since compatibility with existing behavior will be an important part of the
resolution of this issue (truncation of Stream_IO files with mode Out_File),
I've created a test and distributed it to implementors. (If anyone out there
would like to try it themselves, just drop me a line and I'd be happy to send
it. I'm not going to post it to the list as it is 535 lines long.) I'll post a
summary of results in a week or so.

I've run the test on 4 popular compilers for Windows, and I have a few
preliminary conclusions. (I'm writing this up now, rather than waiting for
additional results, so I don't forget all of this over the long holiday
weekend. I expect to have fewer brain cells next Wednesday... :-)

-- Most implementors do in fact expect to have to truncate Stream_IO files on
   Close. Apparently, I'm the only implementor who finds this bizarre.

-- None of the compilers I tested supports Pascal's suggested "workaround",
   including Pascal's compiler. :-) That suggests if this topic is handled with
   a confirmation AI, a strong ACATS test is needed to insure that the
   workaround actually works.

-- When positioning is added to the mix, the results vary wildly. I'll need
   more results to form any opinion here. It appears to be very dangerous to
   open a Stream_IO file with mode Out_File and write only a few elements
   before closing with current compilers.

-- Stream_IO is still buggy: several implementation raise End_Error when
   Stream_IO.Read reads past the end of a file; one implementation deletes the
   entire contents of the file when it is opened with mode Out_File (even if it
   is Set_Moded to In_File and closed without other operations).


It is interesting to note that although implementors mostly believe that
truncation is required on Close and Reset, it is possible to read the RM so
this is not a requirement.

A.12.1(28) says that "... Close, ... Reset, ... has the same effect as the
corresponding subprograms in Sequential_IO."  A.8.2(10) says, "In addition, for
sequential files, if the file being closed has mode Out_File or Append_File,
then the last element written since the most recent open or reset is the last
element that can be read from the file." (The language for Reset in A.8.2(16)
is almost identical.) A.8(1) says that a file used for sequential access is a
sequential file. The AARM also says specificially that stream files are not
sequential files (see A.8(2.a-b)).

So, we can argue both ways:
Since Sequential_IO only works on sequential files, the truncation requirement
also applies to Stream_IO by a literal reading of A.12.1(28).

Alternatively, since the description of Close and Reset say that truncation
only applies to sequential files, and the clear language designer's intent is
that stream files are not sequential files, then the truncation does not apply
to stream files.

Both interpretations seem valid to me, but of course are completely different.


A common implementation of truncation for sequential files (in Sequential_IO)
is to empty the file when it is opened. This is more efficient than truncating
the file on Close (most sequential files are Created, not Opened with Out_File,
and usually no truncation is needed), and it works even on operating systems
that do not support truncating files (because the file can be deleted on Open
to provide the correct semantics).

However, a direct copying of this implementation to stream files leads to
disaster. For instance, if Stream_IO is used to maintain a database, updating a
single record by opening the file with mode Out_File, writing the record, and
closing the file would discard the contents of the entire database! At least
one existing implementation appears to have made this mistake.

It is also worth noting that a correct implementation of Stream_IO truncation
on an operating system that does not support file truncation would be very
expensive (requiring an under-the-covers copy of the file). I'm not sure that
this is a real issue (the OSes that I used that didn't support truncation are
long obsolete), but I think we need to consider it.


Finally, exactly how the truncation is done when positioning is involved is
unclear. Recall the wording of A.8.2(10) says that the "last element written"
is the last one in the file after a Close. Does this mean the "most recent
element written" or "the element with the largest Index written"? A natural
English reading would imply the former; however implementations seem to favor
the latter.


In the absence of compatibility issues, I'd suggest reading the RM to say that
truncation is not required. However, given the trend of implementations, that
probably would be a bad idea. Thus we're going to have to figure out how
something intended solely for sequential files (truncation) applies to files
with positioning.

Sigh. Stream_IO is still a mess.

*************************************************************

From: Randy Brukardt
Date: Monday, February  4, 2002  7:56 PM

You may recall that back in December, I created and distributed a test to
determine the current behavior of Ada compilers when closing or resetting a
stream file with mode Out_File. Here is the promised summary of the
results -- they're pretty grim in that compilers are very inconsistent, even
compilers from the same vendor.

I received results from most of the compiler vendors, but I didn't receive
results from DDCI or from ICC, despite repeated requests. All other
compilers are listed in alphabetical order of the product name.

Aversomething AdaMagic (latest development version):

   -- Truncates sequential-access stream files on close/reset: No.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: No.
   -- Supports Set_Mode "workaround" to avoid truncation: Yes.
   -- Truncates positional-access stream files on close/reset: No.
   -- Discards data on Open for Out_File for positional-access stream files: No.
   -- No Truncation.
   -- File opened with mode Append_File truncates after positioning and writing in middle:
		Positioning not allowed.

Rational Apex for Sun Solaris (latest development version):

   -- Truncates sequential-access stream files on close/reset: Yes.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: Yes.
   -- Supports Set_Mode "workaround" to avoid truncation: No.
   -- Truncates positional-access stream files on close/reset: Yes.
   -- Discards data on Open for Out_File for positional-access stream files: Yes (very bad).
   -- Truncation is at the largest position element written.
   -- File opened with mode Append_File truncates after positioning and writing in middle: No.
   --    (This implementation appears to violate AI-85 by allowing positioning,
   --	  but ignoring the result and writing at the end anyway.)

Rational Apex for Windows NT (3.2.0c):

   -- Truncates sequential-access stream files on close/reset: Yes.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: No.
   -- Supports Set_Mode "workaround" to avoid truncation: No.
   -- Truncates positional-access stream files on close/reset: Yes.
   -- Discards data on Open for Out_File for positional-access stream files: No.
   -- Truncation is at the largest position element written.
   -- File opened with mode Append_File truncates after positioning and writing in middle: No.
   (Implementation incorrectly raises End_Error when reading past the end of the file
    with Stream_IO.Read)

GNAT for Sun Solaris (latest development version):

   -- Truncates sequential-access stream files on close/reset: Yes.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: No.
   -- Supports Set_Mode "workaround" to avoid truncation: No.
   -- Truncates positional-access stream files on close/reset: Yes.
   -- Discards data on Open for Out_File for positional-access stream files: Yes (very bad).
   -- Truncation is at the largest position element written.
   -- File opened with mode Append_File truncates after positioning and writing in middle: No.

GNAT for Windows NT (3.14a1):

   -- Truncates sequential-access stream files on close/reset: Yes.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: No.
   -- Supports Set_Mode "workaround" to avoid truncation: No.
   -- Truncates positional-access stream files on close/reset: Yes.
   -- Discards data on Open for Out_File for positional-access stream files: Yes (very bad).
   -- Truncation is at the largest position element written.
   -- File opened with mode Append_File truncates after positioning and writing in middle: No.

Janus/Ada for Windows NT (3.1.2 development)

   -- Truncates sequential-access stream files on close/reset: No.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: No.
   -- Supports Set_Mode "workaround" to avoid truncation: Yes.
   -- Truncates positional-access stream files on close/reset: No.
   -- Discards data on Open for Out_File for positional-access stream files: No.
   -- No Truncation.
   -- File opened with mode Append_File truncates after positioning and writing in middle: No.

Aonix ObjectAda for Sun Solaris (version 7.2.1)

   -- Truncates sequential-access stream files on close/reset: Yes.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: Yes.
   -- Supports Set_Mode "workaround" to avoid truncation: No.
   -- Truncates positional-access stream files on close/reset: Yes (bizarre behavior: truncates
         to the number of elements written, ignoring the positioning
         --  except for the basic case of immediate Close after writing, which
         works as expected)).
   -- Discards data on Open for Out_File for positional-access stream files: Yes (very bad).
   -- Can't tell truncation point (due to bizarre behavior).
   -- File opened with mode Append_File truncates after positioning and writing in middle: No.
   --    (This implementation appears to violate AI-85 by allowing positioning,
   --     but ignoring the result and writing at the end anyway.)

Aonix ObjectAda for Windows NT (version 7.2.1)

   -- Truncates sequential-access stream files on close/reset: Yes.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: Yes.
   -- Supports Set_Mode "workaround" to avoid truncation: No.
   -- Truncates positional-access stream files on close/reset: Yes (bizarre
         behavior: truncates to the number of elements written, ignoring the positioning).
   -- Discards data on Open for Out_File for positional-access stream files: Yes (very bad).
   -- Can't tell truncation point (due to bizarre behavior).
   -- File opened with mode Append_File truncates after positioning and writing in middle: Yes.

OCS PowerAda/6000 Compiler (Version 4.3)

   -- Truncates sequential-access stream files on close/reset: Yes.
   -- Truncates stream files opened with In_File mode, then Reset to Out_File: Yes.
   -- Supports Set_Mode "workaround" to avoid truncation: No.
         (Set_Mode unexpectedly raised Use_Error at this point [with an Exception_Message
          of "Set_Mode not yet supported"], which terminated the test.)


The main conclusion that can be drawn from this is that there is no
consistency, even between the same vendor's implementations. Probably some
sort of truncation for sequential-access files should be required (since
most vendors do it, even though neither the AARM nor Tucker support the
requirement), but beyond that it is very murky.

OTOH, the large number of implementations that discard data when a file is
Opened with mode Out_File (even if later access is simply an update of a few
elements) is horrifying. It means that Stream_IO cannot be used in
applications requiring in-place updating -- an application that was clearly
intended by the language designers.

Also, hardly any of the implementations support the workaround suggested to
avoid truncation. So, if a user want to avoid this behavior (for in-place
updating, for example), they're out of luck with most implementations.

We'll have to discuss the best approach at the upcoming meeting.

*************************************************************


Questions? Ask the ACAA Technical Agent