Version 1.10 of ais/ai-00085.txt

Unformatted version of ais/ai-00085.txt version 1.10
Other versions for file ais/ai-00085.txt

!standard A.8.2 (16)          05-01-25 AI95-00085/09
!standard A.12.1(28.1)
!standard A.12.1(33)
!class confirmation 95-08-19
!status Amendment 200Y 02-07-09
!status WG9 Approved 02-06-21
!status ARG Approved 6-0-1 02-02-12
!status work item 98-10-09
!status ARG Approved (subject to editorial review) 12-0-0 98-04-03
!status work item 95-11-01
!status received 95-08-19
!priority High
!difficulty Hard
!subject Append_File, Reset, and positioning for Stream_IO
!summary
A call to Reset without the Mode parameter is equivalent to a call to Reset with the current mode of the file.
The beginning and end of the file mentioned in the description of the semantics of Reset designate the beginning and end of the external file.
It is legitimate for an implementation to raise Use_Error on the positioning operations of Stream_IO if the underlying operating system does not support positioning.
The procedure Stream_IO.Set_Mode sets the mode of the file, it doesn't necessarily change it.
!question
1. Are the following two calls equivalent, even if F is opened in
Append_File mode? (Yes.)
Reset (F, Mode => Mode (F)); Reset (F);
2. A call to Reset with mode In_File allows reading to be restarted
"from the beginning of the file". A.7(2) indicates that the term "file" refers to a file object, not to an external file. Does the phrase "beginning of the file" refer to the beginning of the external file? (Yes.)
3. For a Unix implementation that uses the Unix append mode to implement
files opened in Append_File mode, what are the possible effects of Set_Index? (Do the right thing or raise Use_Error.)
4. A.12.1(28.1/1) says that "the Set_Mode procedure changes the mode of
file." Does that mean that Set_Mode can only be used to "change" the mode of the file? (No.)
!response
1. A.8.2(16) states that: "if a Mode parameter is supplied, the current mode of the given file is set to the given mode." This paragraph doesn't mention any other effect for the Mode parameter; therefore, setting the Mode parameter Mode (F) has essentially no effect, and the two calls Reset (File, Mode (File)) and Reset (File) are equivalent.
For a file which is opened in Append_File mode, Reset (F) positions the file so that writing can be restarted after the last element of the external file. This is explicitly stated in A.8.2(16), although this section uses the word "file" when it should say "external file".
Note that for a Unix implementation, it makes it possible to use the Unix append mode, and to implement any of these two calls without closing and reopening the file.
2. When A.8.2(16) talks about the "beginning of the file" or "the last element of the file" it is talking about the beginning and end of the external file (despite A.7(2)).
For a file that is Reset to mode In_File, reading restarts at the beginning of the external file. This is supported by the annotation in AARM A.10.2(4.a), which indicates that Reset should be similar to closing and then reopening. The internal file essentially disappears between closing and reopening, so clearly Reset is resetting based on properties of the external file, not of the internal file.
Note that for a Unix implementation which uses the Unix append mode to implement files opened in Append_File mode, this means that resetting to In_File or Out_File must be implemented by closing and reopening.
3. It is the intent that a Unix implementation can use the Unix append mode to implement files opened in Append_File mode. For a stream designating such a file, positioning is effectively impossible because, if the O_APPEND bit was set at open time, "the file pointer is set to the end of the file prior to each write" (excerpt from the Unix man page for open). A.12.1(33) makes it clear that such an implementation is legitimate, provided that it raises Use_Error on the positioning operations: "if positioning is not supported for the given file, then a call to Index or Set_Index propagates Use_Error."
The standard makes no requirements about whether a particular file supports positioning or not. Thus, an implementation does not have to support positioning on any files at all if it likes. It may use any rule that is appropriate for the target system. In particular, it is acceptable for a file to not support positioning if the mode is Append_File, and for the same external file to support positioning if the mode is Out_File.
Although this compromises portability (different compilers may use different implementations, and certainly different OSes may have different restrictions), the stress here is on performance: the intent is that the file and stream operations should be implemented as a thin layer on top of the OS calls, and should not have to simulate services that are not provided by the underlying OS. Moreover, since there is no requirement to support positioning for any stream files, a truly portable program would have to avoid positioning a stream file in any mode. Thus, allowing implementations to use O_APPEND does not impose any further portability penalty.
4. A.12.1(28.1/1) should say that "the procedure Set_Mode sets the mode of the file." If the Mode parameter is the current mode of the file, then surely the mode of the file is not changed.
!corrigendum A.8.2(16)
Replace the paragraph:
Resets the given file so that reading from its elements can be restarted from the beginning of the file (for modes In_File and Inout_File), and so that writing to its elements can be restarted at the beginning of the file (for modes Out_File and Inout_File) or after the last element of the file (for mode Append_File). In particular, for direct access this means that the current index is set to one. If a Mode parameter is supplied, the current mode of the given file is set to the given mode. In addition, for sequential files, if the given file has mode Out_File or Append_File when Reset is called, the last element written since the most recent open or reset is the last element that can be read from the file. If no elements have been written and the file mode is Out_File, the reset file is empty. If no elements have been written and the file mode is Append_File, then the reset file is unchanged.
by:
Resets the given file so that reading from its elements can be restarted from the beginning of the external file (for modes In_File and Inout_File), and so that writing to its elements can be restarted at the beginning of the external file (for modes Out_File and Inout_File) or after the last element of the external file (for mode Append_File). In particular, for direct access this means that the current index is set to one. If a Mode parameter is supplied, the current mode of the given file is set to the given mode. In addition, for sequential files, if the given file has mode Out_File or Append_File when Reset is called, the last element written since the most recent open or reset is the last element that can be read from the external file. If no elements have been written and the file mode is Out_File, the reset file is empty. If no elements have been written and the file mode is Append_File, then the reset file is unchanged.
!corrigendum A.12.1(28.1/1)
Replace the paragraph:
The Set_Mode procedure changes the mode of the file. If the new mode is Append_File, the file is positioned to its end; otherwise, the position in the file is unchanged.
by:
The Set_Mode procedure sets the mode of the file. If the new mode is Append_File, the file is positioned to its end; otherwise, the position in the file is unchanged.
!appendix

!section A.7(02)
!subject Questions about Append_File mode
!reference RM95-A.7(2)
!reference RM95-A.8.2(15,16)
!reference AARM95-A.10.2(4.a)
!from Keith Thompson 95-08-09
!reference as: 95-5253.a Keith Thompson 95-8-9>>
!discussion

Some aspects of Append_File mode seem unclear.  For example:

1. Reset with an explicit Mode => Append_File (for a text or sequential
   file) causes writing to continue at the end of the file, yes?
   Assume F is a (text or sequential) file whose current mode is
   Append_File.  Are the following two calls equivalent?

      Reset(F, Mode => Append_File);
      Reset(F);

   More generally, are these always equivalent?

      Reset(F, Mode => Mode(F));
      Reset(F);

   I believe they should be, but I've been unable to prove it from the
   RM or AARM.

2. A call to Reset with mode In_File allows reading to be restarted
   "from the beginning of the file".  RM95-A.7(2) indicates that
   the term "file" refers to a file object, not to an external file.
   Does the phrase "beginning of the file" refer to the beginning
   of the external file?

   In particular, what is the effect of the following code fragment,
   assuming all operations are successful?

      declare
         package Int_IO is new Ada.Sequential_IO(Integer);
         F: Int_IO.File_Type;
         File_Name: constant String := "foo.dat";
      begin
         Int_IO.Create(F, File_Name);
         for I in 1 .. 3 loop
            Int_IO.Write(F, I);
         end loop;
         Int_IO.Close(F);
         --
         -- foo.dat now contains (1, 2, 3)
         --
         Int_IO.Open(F, Mode => Int_IO.Append_File, Name => File_Name);
         for I in 4 .. 6 loop
            Int_IO.Write(F, I);
         end loop;
         --
         -- foo.dat now contains (1, 2, 3, 4, 5, 6),
         -- but where is the "beginning of the file"?
         --
         Int_IO.Reset(F, Mode => Int_IO.Out_File);
         Int_IO.Write(F, 999);
         Int_IO.Close(F);
      end;

   If the "beginning of the file" is the beginning of the external
   file, the final contents of foo.dat should be the single integer 999.
   If it refers to the beginning of the file object, *and* the beginning
   of a file in Append_File mode is considered to be the point at which
   it was opened, foo.dat should contain 4 integers: 1, 2, 3, and 999.
   (The previously written values 4, 5, and 6 are lost in either case).

   It would be easier and more intuitive, IMHO, for Reset to go to the
   beginning of the external file, but I can't infer this behavior from
   the RM.  The note in AARM95-A.10.2(4.a):

      The behavior of Reset should be similar to closing a file and
      reopening it with the given mode.

   tends to support this, but it's not binding.

   On the other hand, if Reset goes back to the beginning of the external
   file, there's no easy way to go back to the position at which it was
   opened for appending.  (This is probably no great loss.)

****************************************************************

!section A.8.2(16)
!subject Questions about Append_File mode
!reference AI95-00085
!reference RM95-A.8.2(16)
!reference RM95-A.7(2)
!reference AARM-A.10.2(4.a)
!from Tucker Taft 95-10-30
!reference 95-5373.h Tucker Taft 95-10-30>>
!discussion

I certainly believe that Reset(File, Mode(File)) is equivalent
to simply Reset(File).  I can't imagine any other meaningful
interpretation for A.8.2(16).

As far as external vs. internal file, I believe that
the wording in A.8.2(16) is sloppy, and whenever it talks
about the "beginning of the file" or "the last element
of the file" it is talking about the beginning and end of the
external file (despite A.7(2)).  It makes no difference whether
the file was initially opened in append mode, or was reset to
that mode somewhere along the way.

The annotation in A.10.2(4.a) seems to make this intent clear,
by indicating that Reset should be similar to closing and then
reopening.  The internal file essentially disappears between
closing and reopening, so clearly Reset is resetting based on
properties of the external file, not of the "internal" file.

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 8:36 AM
Subject: 	Re: AI95-00085

<<It is legitimate for an implementation to raise Use_Error on the
positioning operations of Stream_IO in the circumstances where the
underlying operating system does not support positioning.
>>

I find this FAR too vague. Obviously if poisitioning is not supported
at all (a system with only tapes :-) then no argument.

But how about a unix-like system in which general positioning is of
course supported, but you *prefer* to use some specific OS facility
to implement e.g. append mode files, which does not support the
positioning that is needed. Are you allowed to blow away the call
with a use_error. I think the answer should be no!

There should be maximum pressure to make your IO system conform to the
Ada model, and I dislike strong suggestions to the contrary. I think
this paragraph should be removed from the AI. It adds nothing, except
an unwelcome suggestion that this kind of compromise is perfectly OK.

****************************************************************

From: 	Pascal Leroy[SMTP:phl@Rational.Com]
Sent: 	Thursday, April 09, 1998 10:55 AM
Subject: 	Re: AI95-00085

> But how about a unix-like system in which general positioning is of
> course supported, but you *prefer* to use some specific OS facility
> to implement e.g. append mode files, which does not support the
> positioning that is needed. Are you allowed to blow away the call
> with a use_error. I think the answer should be no!

But the feeling at the last meeting was precisely that the implementation
should be allowed to use Unix O_APPEND, and that if it does so it probably has
to raise Use_Error on a call to Set_Index for a stream that is open in
Append_Mode.

It seems to me that requiring an implementation to support Set_Index on a
stream in Append_Mode effectively precludes use of O_APPEND, and that's
annoying.

Pascal

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 9:10 AM
Subject: 	Re: AI95-00085

<<It seems to me that requiring an implementation to support Set_Index on a
stream in Append_Mode effectively precludes use of O_APPEND, and that's
annoying.
>>

Why is that annoying?


If people want to use Unix-specific I/O, please do it directly! The whole
idea in defining an I/O facility at the language level is for portability.
Yes, it is frustrating when the models do not match, but that's too bad,
since models between operating systems do not match precisely, this is
inevitable. I think the decision to allow O_APPEND here is plain wrong.
It is perfectly possible to implenment the Ada I/O facilities without
allowing O_APPEND.

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 9:15 AM
Subject: 	Re: AI95-00085

Incidentally, I absolutely agree that the design of the Ada with respect
to Append was injudicious. I suspect that whoever did the design simply
did not take the effort to look at the Unix semantics.

But there are only two acceptable approaches in my mind at this stage:

1. Change the language to match the Unix semantics

2. Insist on the (perfectly possible to meet even on Unix) requirement
that the implementation implement the language as chosen.

The permission not to implement things in the RM is supposed to handle
cases where it is impractical or impossible to implement things right,
not simply cases where implementing things right is, to use Pascal's
word, "annoying".

****************************************************************

From: 	Pascal Leroy[SMTP:phl@Rational.Com]
Sent: 	Thursday, April 09, 1998 11:43 AM
Subject: 	Re: AI95-00085

> 1. Change the language to match the Unix semantics

Well, if you ask me, the notion of calling Set_Index on a stream in
Append_Mode if pretty weird, and it's not clear what the semantics of such a
call should be.  Say that after calling Set_Index, I call Write: what happens?
 Does it write at the end of the file?  At the current point?  Does it
truncate the file?

At the very least, these questions require clarification, and this is not
changing the language.  I am tempted to say that Set_Index in Append_Mode is
likely to be a programming error, and should always raise some exception (and
I now regret that I didn't write the AI in this direction in the first place).

> 2. Insist on the (perfectly possible to meet even on Unix) requirement
> that the implementation implement the language as chosen.
>
> The permission not to implement things in the RM is supposed to handle
> cases where it is impractical or impossible to implement things right,
> not simply cases where implementing things right is, to use Pascal's
> word, "annoying".

I was not thinking in terms of implementation, but in terms of usage.  We have
many users who want to mix Ada and C I/Os on the same file.  I would find it
rather unpleasant if an Ada Append_Mode file was not opened with O_APPEND, or
if passing an O_APPEND file to Ada caused it to be closed and reopened so that
Ada can do its silly simulation.

Pascal

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 9:46 AM
Subject: 	Re: AI95-00085

<<I was not thinking in terms of implementation, but in terms of usage.  We have
many users who want to mix Ada and C I/Os on the same file.  I would find it
rather unpleasant if an Ada Append_Mode file was not opened with O_APPEND, or
if passing an O_APPEND file to Ada caused it to be closed and reopened so that
Ada can do its silly simulation.
>>

Mixing Ada and C I/O may or may not succeed. It is not a requirement that
it work smoothly, just something nice to have. You are free to implement
a form parameter to ensure this smooth interchange. As I am sure you know
GNAT takes a different tack, we allow a file to be opened using the stream
identifier passed in from C, and that ensures a common view of the file.

****************************************************************

From: 	Robert I. Eachus[SMTP:eachus@mitre.org]
Sent: 	Thursday, April 09, 1998 10:10 AM
Subject: 	Re: AI95-00085

At 10:15 AM 4/9/98 EDT, Robert Dewar wrote:
>The permission not to implement things in the RM is supposed to handle
>cases where it is impractical or impossible to implement things right,
>not simply cases where implementing things right is, to use Pascal's
>word, "annoying".

   Is an implementation allowed to silently close and reopen an (external)
file in response to a Set_Position, if it was originally opened in Append
mode?  I assume so, and that this would be the preferable option for Unix.
(I'm not sure what the right thing to do is if an unnamed file/pipe was
opened in append mode though.  Probably, that is a case for Use_Error.)

                                        Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

****************************************************************

From: 	Tucker Taft
Sent: 	Thursday, April 09, 1998 10:42 AM
Subject: 	Re: AI95-00085

Another possibility would be to use O_APPEND mode until a
Set_Index is called, and only then go through the trouble of
re-opening in non-append mode rather than raising Use_Error,
with the presumption that Set_Index will be used rarely for
files opened in append mode.
Effectively Set_Index would be almost like a Reset(F, Mode => Out_File)
(though of course Mode(F) would still return Append_File).
-Tuck

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 11:50 AM
Subject: 	Re: AI95-00085

<<Another possibility would be to use O_APPEND mode until a
Set_Index is called, and only then go through the trouble of
re-opening in non-append mode rather than raising Use_Error,
with the presumption that Set_Index will be used rarely for
files opened in append mode.
Effectively Set_Index would be almost like a Reset(F, Mode => Out_File)
(though of course Mode(F) would still return Append_File).
>>

That's perfectly reasonable. We do something like this in GNAT when you
switch from read to write mode for stream_io, we first open in read only
mode, then reopen in update mode when you switch.

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 11:47 AM
Subject: 	Re: AI95-00085

<<   Is an implementation allowed to silently close and reopen an (external)
file in response to a Set_Position, if it was originally opened in Append
mode?  I assume so, and that this would be the preferable option for Unix.
(I'm not sure what the right thing to do is if an unnamed file/pipe was
opened in append mode though.  Probably, that is a case for Use_Error.)
>>

Of course you are allowed to silently close and reopen in response to
Set_Position (you can do it on every read if you like :-)

****************************************************************

From: 	Randy Brukardt[SMTP:Randy@rrsoftware.com]
Sent: 	Thursday, April 09, 1998 12:29 PM
Subject: 	RE: AI95-00085

I've been following this discussion, and I just don't get it.  Probably the
problem is my lack of experience with the common usages of Unix.

I just don't understand why anyone would want to use O_APPEND on a
positionable device.  When I studied the Unix documents, I just viewed
O_APPEND as (to borrow Robert's term) a
junk mode.  Why would anyone WANT to disallow positioning on their file?  I
can see that it may not matter in some cases, but I can't see any reason
that anyone would want the defined behavior of O_APPEND.  I would expect
that most users just want to start writing at the end of the file, and
don't care about positioning.

I would expect that non-positionable devices would essentially be in append
mode at all times, so there is no need for some junk mode in that case,
either.

The Ada Append_Mode matches the behavior I would expect exactly: starting
writing at the end of the file, but has no other semantic effects.  If this
means not using a junk mode, so be it.

It was always my view that Append is a feature of Open/Create, not a mode.
 It should have been implemented as an optional parameter to Open/Create.

As far as mixing Ada and C I/O, I don't see a problem.  If the file is
opened in C, and imported into Ada somehow, there cannot be a problem.
 That is outside of anything in the Ada standard, so do whatever you want -
raise Use_Error anywhere you have to.  If the file is opened in Ada and
passed to C, there shouldn't a be a problem.  Since O_APPEND disallows
various operations, I don't see how not using it on a file could break any
existing C code (unless that C code was completely broken in the first
place).  In any, I would expect most code that takes files to work
irregardless of the file mode.

If you absolutely have to have O_APPEND opens, add a form parameter
"Open_with_junk_O_Append" and open it that way.  Then you can raise
Use_Error anywhere you like.

I agree with Robert that there is no good reason to break portability of
software by allowing compilers to raise Use_Error when it is easy to avoid
it.  This is especially so in order to support junk file options of a
particular (admittedly popular) operating system.


						Randy.

****************************************************************

From: 	Tucker Taft[SMTP:stt@inmet.com]
Sent: 	Thursday, April 09, 1998 2:14 PM
Subject: 	RE: AI95-00085

> I've been following this discussion, and I just don't get it.  Probably the
> problem is my lack of experience with the common usages of Unix.
>
> I just don't understand why anyone would want to use O_APPEND on a
> positionable device.  When I studied the Unix documents, I just viewed
> O_APPEND as (to borrow Robert's term) a
> junk mode.  Why would anyone WANT to disallow positioning on their file?  I
> can see that it may not matter in some cases, but I can't see any reason
> that anyone would want the defined behavior of O_APPEND.  I would expect
> that most users just want to start writing at the end of the file, and
> don't care about positioning.

O_APPEND is a relatively new mode for Unix (speaking geologically here).
It was invented to solve the problem of having multiple processes
all writing to the same log file at the same time, wanting all messages
to end up at the end, no matter which process wrote them.

Without O_APPEND, each process maintains its own internal idea of what
is its position in the file, and even though it may position to the
end when it first opens the file, because other processes are writing
to the same file simultaneously, its position may no longer correspond
to the end when it gets around to writing.

In my view, the "Append_File" mode in Ada is intended to allow the use
of O_APPEND, but not require it, at least for Sequential_IO.
However, A.14 makes it clear that each stream file object has its
own current_index, presuming the file supports positioning.
This would seem to preclude the use of O_APPEND, with its automatic
position-to-the-end semantics, unless positioning is disallowed
for the file.   Hence it seems necessary to either raise Use_Error
on Set_Index, or not use O_APPEND mode for stream files.
Presumably a Form parameter could be used to control which
semantics are preferred.

> I would expect that non-positionable devices would essentially be in append
> mode at all times, so there is no need for some junk mode in that case,
> either.
>
> The Ada Append_Mode matches the behavior I would expect exactly: starting
> writing at the end of the file, but has no other semantic effects.  If this
> means not using a junk mode, so be it.
>
> It was always my view that Append is a feature of Open/Create, not a mode.
>  It should have been implemented as an optional parameter to Open/Create.

The issue is what happens in the presence of multiple processes all
appending to the same file.

> I agree with Robert that there is no good reason to break portability of
> software by allowing compilers to raise Use_Error when it is easy to avoid
> it.  This is especially so in order to support junk file options of a
> particular (admittedly popular) operating system.

It seems better to support either approach by use of a Form parameter.
It is not clear to me which should be the default for Append_File
in the absence of a Form parameter, but it now seems clear to me
that O_APPEND mode is inconsistent with the A.14(4) requirement
that each stream file maintains its own current index, unless
positioning is not allowed for the file (i.e. Set_Index raises Use_Error).

> 						Randy.

-Tuck

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Thursday, April 09, 1998 2:41 PM
Subject: 	Re: AI95-00085

This kind of Ada-centric view (of append-only file access modes)
is typical of the kind of attitude that has held Ada back.

The Ada'95 append mode was intended to allow access to OS-provided
append modes.  If we now prevent it from mapping in a sensible
way, we just make Ada look absurd.

--Ted

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 2:40 PM
Subject: 	Re: AI95-00085

<<This kind of Ada-centric view (of append-only file access modes)
is typical of the kind of attitude that has held Ada back.

The Ada'95 append mode was intended to allow access to OS-provided
append modes.  If we now prevent it from mapping in a sensible
way, we just make Ada look absurd.
>>

Are you specifically talking about O_APPEND here?

We may say that the Ada 95 append mode was intended to allow access to
OS-provided append modes, but all OS's do not provide the same stuff, are
you saying that you think append mode should be entirely implementation
dependent?

****************************************************************

From: 	Randy Brukardt[SMTP:Randy@rrsoftware.com]
Sent: 	Thursday, April 09, 1998 3:16 PM
Subject: 	RE: AI95-00085

>O_APPEND is a relatively new mode for Unix (speaking geologically here).
>It was invented to solve the problem of having multiple processes
>all writing to the same log file at the same time, wanting all messages
>to end up at the end, no matter which process wrote them.

OK.  Sounds like my impression is correct: it is a junk mode created to
patch around the lack of kernel-level sharing, good synchronization
methods, and shared address space threads in [basic] Unix, and a lack of
awareness of tasking issues in C.  Such stuff is totally unnecessary from
an Ada perspective, which has good synchronization methods and tasking.

This just makes it a C-to-Ada interface issue, and such issues are best
handled with forms and pragmas, not by messing with the basic Ada
semantics.

					Randy.

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Thursday, April 09, 1998 4:10 PM
Subject: 	Re: AI95-00085

| Of course you are allowed to silently close and reopen in response to
| Set_Position (you can do it on every read if you like :-)

The above would be very unwise in a UNIX system, since the semantics
would be much different than if the file were to stay open.  In particular,
a file that is open cannot go away, even if it is (concurrently) unlinked,
whereas once an unlinked file is closed it goes away.

I suppose this is another respect in which the ARM semantics don't match
UNIX semantics.  The ARM says that when you Delete a file it ceases to exist.
In UNIX, a file does not cease to exist until the last close.

I can't imagine that existing Ada implementations on UNIX systems are
required to somehow defeat the UNIX last-close semantics, so I guess the
requirement about files ceasing to exist immediately after Delete is
not being enforced.

Given this, why not leave the Append semantics open enough that a direct
mapping to O_APPEND mode is OK?

--Ted

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Thursday, April 09, 1998 4:24 PM
Subject: 	RE: AI95-00085

| I just don't understand why anyone would want to use O_APPEND ...
| ... junk mode.  ... I can't see any reason
| that anyone would want the defined behavior of O_APPEND. ...

O_APPEND serves a critical purpose, i.e., a safe way to write to
log-files by multiple processes.  When you use it, you DON'T want
to allow positioning the file, and the mode prevents you from
doing so; however, the file normally is disk file and so
positionable if opened in other modes.

[In case it is not sufficiently obvious, O_APPEND does something
you *cannot* do with lseek() to the end of the file, because the
lseek() and the following write operation are not atomic, and so
one process could end up writing on top of something just
previously logged by another.]

| It was always my view that Append is a feature of Open/Create, not a mode.
| It should have been implemented as an optional parameter to Open/Create.

??????????  Append *is* something you specify in Open/Create.

--Ted

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Thursday, April 09, 1998 4:31 PM
Subject: 	Re: AI95-00085

| Are you specifically talking about O_APPEND here?

Certainly it should include O_APPEND, or more specifically the
append-only (nonrepositionable, always-write-at the end) semantics.
Otherwise, if the semantics could be emulated using other existing
Ada operations, there would be no reason to add an append mode
to the language.

| We may say that the Ada 95 append mode was intended to allow access to
| OS-provided append modes, but all OS's do not provide the same stuff, are
| you saying that you think append mode should be entirely implementation
| dependent?

Not "entirely", but certainly loose enough to allow mapping to O_Append.

--Ted

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Thursday, April 09, 1998 4:27 PM
Subject: 	RE: AI95-00085

| However, A.14 makes it clear that each stream file object has its
| own current_index, presuming the file supports positioning.
| This would seem to preclude the use of O_APPEND, with its automatic
| position-to-the-end semantics, unless positioning is disallowed
| for the file.   Hence it seems necessary to either raise Use_Error
| on Set_Index, or not use O_APPEND mode for stream files.

Presumably the only issue here is whether we want people to be able
to use stream I/O to write log files from Ada programs in a concurrent
environment.  I've been presuming everyone would agree that the answer
is "yes".

--Ted

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 4:45 PM
Subject: 	RE: AI95-00085

<<O_APPEND serves a critical purpose, i.e., a safe way to write to
log-files by multiple processes.  When you use it, you DON'T want
to allow positioning the file, and the mode prevents you from
doing so; however, the file normally is disk file and so
positionable if opened in other modes.
>>

It is inappropriate to expect the Ada feature to provide this functionality.
It is not documented to do so, and relying on it to do so, because you know
your implementation happens to work this way is a style of non-portable
programming that is not to be encouraged. If you want to rely on a Unix
specific feature, use the Unix specific feature directly!

<<??????????  Append *is* something you specify in Open/Create.
>>

No, it is a mode, and can be reset -- I very much agree that it should have
been an optional parameter of Open/Create, but it is not!

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 4:42 PM
Subject: 	Re: AI95-00085

<<I can't imagine that existing Ada implementations on UNIX systems are
required to somehow defeat the UNIX last-close semantics, so I guess the
requirement about files ceasing to exist immediately after Delete is
not being enforced.
>>

Certainly GNAT does reopens in some cases, precisely to get a BETTER
match to Unix semantics (i.e. do not open in update mode until you know
you are going to be writing).

<<Given this, why not leave the Append semantics open enough that a direct
mapping to O_APPEND mode is OK?
>>

Because it introduces unnecessary non-portability.

If you think that it is fine not to be able to position, then your
implementation can use O_APPEND *until* you reposition.

If you want to use O_APPEND and reposition you are out of luck anyway!

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 4:46 PM
Subject: 	RE: AI95-00085

<<Presumably the only issue here is whether we want people to be able
to use stream I/O to write log files from Ada programs in a concurrent
environment.  I've been presuming everyone would agree that the answer
is "yes".
>>

Absolutely NOT!

If you want to rely on Unix specific features for writing log files in
a concurrent safe manner, then you should use the C streams and specific
C/Unix features to do this. You can still use stream attributes in Ada
to write and read the data. I see no particular merit in twisting Stream_IO
into service for this purpose, since the resulting code would be annoyingly
and obscurely implementation dependent.

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 4:52 PM
Subject: 	RE: AI95-00085

<<Pardon me, but please refrain from calling "jun"k that which you
simply may not understand.  It is precisely "kernel-level sharing"
and "good synchronization" that is provided by O_APPEND. The
kernel insures that all writes to the file are at the end, thereby
enforcing data integrity, in a way that otherwise is impossible.
>>

What he means by junk (and I concur with his choice of the term) is some
operating specific special feature meant to solve some problem in a peculiar
operating system idiosyncratic feature. This feature may be interesting for
Unix users, but it has nothing to do with Ada, and *allowing* (rather than
insisting, which is impractical) implementations to map a specific restricted
style of Ada stream_io (without a form parameter) to this strange Unix feature
is not only not helpful it is actively harmful.

If an implentor wants to provide this feature, the proper way to do it is
with a form parameter, that specifies O_APPEND mode is to be used, and then
of course you are free to enforce any restrictions you like.

This is a perfectly valid approach, it makes the implementation dependence
apparent, which is a GOOD THING, while preserving the capability you think
is important in this specific environment.

****************************************************************

From: 	Norman H Cohen[SMTP:ncohen@us.ibm.com]
Sent: 	Thursday, April 09, 1998 4:52 PM
Subject: 	Re: AI95-00085

It seems quite reasonable to assert that repositioning a file opened in append
mode raises Use_Error, and to state that this happens portably and predictably
for all implementations (Write Once, Blow Up Anywhere).  This would allow Ada
append mode to be directly mapped to Unix O_APPEND.  More importantly,
appending and repositioning are a logically inconsistent combination.

-- Norman

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Thursday, April 09, 1998 4:58 PM
Subject: 	Re: AI95-00085

<<It seems quite reasonable to assert that repositioning a file opened in append
mode raises Use_Error, and to state that this happens portably and predictably
for all implementations (Write Once, Blow Up Anywhere).  This would allow Ada
append mode to be directly mapped to Unix O_APPEND.  More importantly,
appending and repositioning are a logically inconsistent combination.
>>

It may be reasonable.

It is a definite language change.

It is perfectly reasonable to append, write stuff, go back and patch something
you already wrote, position to the end, write some more, etc. so I disagree
with the last statement.

****************************************************************

From: 	Jean-Pierre Rosen[SMTP:rosen.adalog@wanadoo.fr]
Sent: 	Friday, April 10, 1998 2:19 AM
Subject: 	Re: AI95-00085

After reading the flood on this subject, I strongly concur with Robert's position, that I would rephrase this way:

Those who want the semantics of a particular OS use the binding to this OS (and we do have a good binding to POSIX).
Those who want portability across various OSes use Ada predefined services.

Ted, are you discovering that portability sometimes requires the least common demoninator of various OSes ?
----------------------------------------------------------------------------
                  J-P. Rosen (Rosen.Adalog@wanadoo.fr)
      Visit Adalog's web site at http://perso.wanadoo.fr/adalog

****************************************************************

From: 	Pascal Leroy[SMTP:phl@Rational.Com]
Sent: 	Friday, April 10, 1998 5:46 AM
Subject: 	Re: AI95-00085

Wow!  30+ messages in my mailbox this morning on this wretched AI!  We have
not had such a lively discussion on the ARG list for a long time.

For the record, I agree with essentially everything that Ted said.

> <<It seems quite reasonable to assert that repositioning a file opened in
> append
> mode raises Use_Error, and to state that this happens portably and
> predictably
> for all implementations (Write Once, Blow Up Anywhere).  This would allow
Ada
> append mode to be directly mapped to Unix O_APPEND.  More importantly,
> appending and repositioning are a logically inconsistent combination.
> >>
>
> It may be reasonable.

I agree with Norm that appending and repositioning are probably inconsistent,
and at the very least don't have a "natural" semantics.  So if nothing else,
this AI _must_ define what repositioning might mean for an append stream.

> It is a definite language change.

Definitely not.  It is very much unclear what is the semantics of
repositioning on an append file in the first place, and the RM doesn't give
any clue as to what it should mean in Ada.  (In fact, it's hard to find a
section of the RM which is less precise than A.12.1.)

So whatever we decide is not a language change, only a clarification.

> It is perfectly reasonable to append, write stuff, go back and patch
> something
> you already wrote, position to the end, write some more, etc. so I disagree
> with the last statement.

That's one possible interpretation.  Here is another one:

On a stream opened in Append_Mode, Set_Index truncates the file at the
specified position, and the next write will take place at the (new) end of the
file.  Surely this preserve two invariants: (1) Set_Index effectively moves to
the specified position and (2) in Append_Mode, you only write to the end of
the file.

These two interpretations are (1) legitimate (2) not substantiated by the RM
and (3) not contradicted by the RM.

Pascal

_____________________________________________________________________
Pascal Leroy                                    +33.1.30.12.09.68
pleroy@rational.com                             +33.1.30.12.09.66 FAX

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Friday, April 10, 1998 6:03 AM
Subject: 	Re: AI95-00085

Pascal says

  Definitely not.  It is very much unclear what is the semantics of
  repositioning on an append file in the first place, and the RM doesn't give
  any clue as to what it should mean in Ada.  (In fact, it's hard to find a
  section of the RM which is less precise than A.12.1.)

  So whatever we decide is not a language change, only a clarification. >>

Robert replies:

  Well it is clear that positioning *is* allowed, so an AI that says it is
  not allowed is clearly a language change.

Pascal suggests a possible interpretation of positioning:

  On a stream opened in Append_Mode, Set_Index truncates the file at the
  specified position, and the next write will take place at the (new) end of the
  file.  Surely this preserve two invariants: (1) Set_Index effectively moves to
  the specified position and (2) in Append_Mode, you only write to the end of
  the file.

  These two interpretations are (1) legitimate (2) not substantiated by the RM
  and (3) not contradicted by the RM.

Robert replies

  I disagree that the RM is unclear, and I do not think that Pascal's
  interpretation here is supportable. The RM simply says about Append_File
  that it causes the first transfer to be at the end of the file.

  I think what is going on here is that people read about append, and
  assume that for stream files it has something to do with O_APPEND in
  Unix, and then they start importing irrelevant semantics from Unix.

  When I looked at the RM, it was immediately clear to me that of course
  you can't use Unix append mode for stream_IO Append_File, since it has
  inconsistent semantics.

  We implemented a long time ago what seemed like the obviously required
  semantics, no one has ever complained, and no ACVC tests has contradicted
  this. I would be *very* reluctant to swich to an interpretation like the
  one above, which has no justification whatsoever in the RM, and might
  well cause silent malfunction of existing programs.

  Since Stream_IO is part of the core, there are 70 or so compilers that
  are implementing some interpretation or other of how this works. Can
  we please do a reasonably comprehensive survey of what implementors
  are in fact doing now, to at least make sure that whatever the ARG
  does in this non-critical area is not unnecesarily disruptive. I say
  non-critical here, because I have not seen users get worried about
  this issue!

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Friday, April 10, 1998 8:28 AM
To: 	arg95@sw-eng.falls-church.va.us
Subject: 	copy of Baker/Dewar exchange


Robert Dewar has requested that I forward the following e-mail exchange
between him and myself, which I intended as a private exchange, but
Robert apparently intended for the whole ARG.
My apologies for the length.  --Ted

---------------------------------------------

From baker Thu Apr  9 18:59:59 1998
To: dewar@gnat.com
Subject: RE: AI95-00085
Content-Length: 556

It is unfair to cite me as a satisfied user of that GNAT idiosyncracy.
My "use" of GNAT is primarily for implementing the POSIX.5 interfaces,
and implementing the GNAT runtime.  This feature does not come up there.
That does not mean it is not an important functionality to real users.

What really galls me about this discussion is that the reason for Append
being added to Ada'9X was input from the POSIX Ada bindings group, and it
specifically was intended to map O_APPEND --- despite whatever revisionist
rhetoric you and Randy want to put out.

--Ted

From dewar@gnat.com Thu Apr  9 23:01 EST 1998
Return-Path: <dewar@gnat.com>
Received: from cs.fsu.edu by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4)
	id XAA20500; Thu, 9 Apr 1998 23:01:25 GMT
Received: from sw-eng.falls-church.va.us (ns1.sw-eng.falls-church.va.us [199.75.54.2]) by cs.fsu.edu (8.8.8/8.7.3) with ESMTP id TAA22871 for <baker@cs.fsu.edu>; Thu, 9 Apr 1998 19:01:27 -0400 (EDT)
Received: from nile.gnat.com by sw-eng.falls-church.va.us (8.8.8/)
	id WAA11205; Thu, 9 Apr 1998 22:57:37 GMT
Received: by nile.gnat.com (5.0/1.20)
	id AA28031; Thu, 9 Apr 98 18:59:04 EDT
Date: Thu, 9 Apr 98 18:59:04 EDT
From: dewar@gnat.com (Robert Dewar)
Message-Id: <9804092259.AA28031@nile.gnat.com>
To: Arg95@sw-eng.falls-church.va.us, baker@dad.cs.fsu.edu, dewar@gnat.com,
        ncohen@us.ibm.com
Subject: Re: AI95-00085
Content-Type: text
Content-Length: 393
Status: RO

<<If that is what you want to do, you don't need an append mode.
You just open the file in write mode, and position to the end when you want
to write at the end.  There is no justification for having an append mode
if the semantics do not provide any additional capability.
>>

That's odd, the whole discussion is about *taking away* functionality from
the current append semantics in the RM!

From baker Thu Apr  9 19:15:45 1998
To: dewar@gnat.com
Subject: Re: AI95-00085
Content-Length: 592

| That's odd, the whole discussion is about *taking away* functionality from
| the current append semantics in the RM!

The capability that is lost from UNIX is (atomic) writes to the end
of a file by multiple processes.  Providing that requires mapping
directly to the OS O_APPEND mode, if there is one.  If we require the
Ada implementation to provide some other functionality (that can be
simulated via other existing operations, and so is not a real gain)
that required simulation of normal append mode writes via some non-atomic
combination of system calls, then there is a loss.

--Ted

From dewar@gnat.com Thu Apr  9 23:05 EST 1998
Return-Path: <dewar@gnat.com>
Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4)
	id XAA20521; Thu, 9 Apr 1998 23:05:13 GMT
Received: by nile.gnat.com (5.0/1.20)
	id AA02217; Thu, 9 Apr 98 19:03:03 EDT
Date: Thu, 9 Apr 98 19:03:03 EDT
From: dewar@gnat.com (Robert Dewar)
Message-Id: <9804092303.AA02217@nile.gnat.com>
To: baker@dad.cs.fsu.edu, dewar@gnat.com
Subject: RE: AI95-00085
Content-Type: text
Content-Length: 795
Status: R

<<What really galls me about this discussion is that the reason for Append
being added to Ada'9X was input from the POSIX Ada bindings group, and it
specifically was intended to map O_APPEND --- despite whatever revisionist
rhetoric you and Randy want to put out.
><>

This is not historically accurate, there were a number of revision requests
for append mode, and they did not mention O_APPEND (I am not even sure it
existed then!)

What puzzles me Ted, is you had years to see a plain definition that was
obviously inconcistent with O_APPEND semantics, and so did everyone else
have this chance. The inconsistency was mentioned several times during
the revision process. No one seemed concerned. No one raised the issue.
It is a bit late to be suggesting significant changes to the language!

From baker Thu Apr  9 19:19:28 1998
To: dewar@gnat.com
Subject: RE: AI95-00085
Content-Length: 1006

| This is not historically accurate, there were a number of revision requests
| for append mode, and they did not mention O_APPEND (I am not even sure it
| existed then!)

It certainly has existed, for many years, and has been in the
POSIX spec. from the very beginning.  The requests where I knew
the people who put them in (like Dave Emery) were motivated by
experience with UNIX/POSIX applications where the functionality
was needed.  Those requests may not have mentioned POSIX
explicitly, so you may not have seen the connection.

| What puzzles me Ted, is you had years to see a plain definition that was
| obviously inconcistent with O_APPEND semantics, and so did everyone else
| have this chance. The inconsistency was mentioned several times during
| the revision process. No one seemed concerned. No one raised the issue.
| It is a bit late to be suggesting significant changes to the language!

Like several other things outside the tasking/AnnexD/AnnexC domain,
I managed to miss this.

--Ted

From dewar@gnat.com Thu Apr  9 23:22 EST 1998
Return-Path: <dewar@gnat.com>
Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4)
	id XAA20558; Thu, 9 Apr 1998 23:22:20 GMT
Received: by nile.gnat.com (5.0/1.20)
	id AA21075; Thu, 9 Apr 98 19:20:09 EDT
Date: Thu, 9 Apr 98 19:20:09 EDT
From: dewar@gnat.com (Robert Dewar)
Message-Id: <9804092320.AA21075@nile.gnat.com>
To: baker@dad.cs.fsu.edu, dewar@gnat.com
Subject: Re: AI95-00085
Content-Type: text
Content-Length: 1558
Status: RO

<<The capability that is lost from UNIX is (atomic) writes to the end
of a file by multiple processes.  Providing that requires mapping
directly to the OS O_APPEND mode, if there is one.  If we require the
Ada implementation to provide some other functionality (that can be
simulated via other existing operations, and so is not a real gain)
that required simulation of normal append mode writes via some non-atomic
combination of system calls, then there is a loss.
>>

Wait a minute, where is the loss here? There is nothing in the RM that
talks about atomic writes by multiple processes. Such multiple writes
to a file are clearly undefined in the Ada semantics.

I really don't understand Ted's position at all. It seems to be something
like:

  a) weaken the Ada semantics so that Ada compilers can use O_APPEND by
	default.

  b) do not require this weakening

  c) do not require the use of O_APPEND

  d) if you are lucky enough to have a compiler that takes advantage of
     this new functionality, then feel free to use it to write
     non-portable code.

Seems quite strange to me. Ted, why do you object to a Form parameter
something like Atomic_Append, which we all agree MUST have the semantics
you want if it is implemented, and is rejected otherwise.

Incidentally, I think Ted's history is totally wrong here.

Append is seriously useful functionality for Sequential and Text_IO. Its
provision in Stream_IO seems to have been mostly a matter of (possibly
misguided) orthogonality.

At least that is the way that Dewar remembers history!

From baker Fri Apr 10 08:02:15 1998
To: dewar@gnat.com
Subject: Re: AI95-00085
Content-Length: 2276

Sorry to spend so much talk on such a minor matter, but not
allowing a mapping to O_APPEND still seems fundamentally
wrong-headed to me, your repeated assertions notwithstanding.
I guess we are not going to agree on this one.

| Wait a minute, where is the loss here?

Loss of permission to access to underlying OS functionality.

|   a) weaken the Ada semantics so that Ada compilers can use O_APPEND by
| 	default.

Weaken here means giving up a functionality (repositioning an
append-only stream) that seems to have no practical use.

|   b) do not require this weakening

Seems to mean the same thing as below.

|   c) do not require the use of O_APPEND

In case the OS does not support O_APPEND.

|   d) if you are lucky enough to have a compiler that takes advantage of
|      this new functionality, then feel free to use it to write
|      non-portable code.

I'd like to see this be the required semantics, for UNIX systems,
e.g. by the POSIX Ada bindings.

| Seems quite strange to me. Ted, why do you object to a Form parameter
| something like Atomic_Append, which we all agree MUST have the semantics
| you want if it is implemented, and is rejected otherwise.

Does GNAT support such a form parameter?
POSIX.5 defined an interface for this, before Ada'9X, but I have not
seen the POSIX.5 form parameters supported.
Moreover, last time we revised POSIX.5, people argued that we did not
need this (complicated and unwieldy) form parameter business any more,
given Ada'95 support for Append mode.

In contrast to the form parameter, O_APPEND is very simple to
implement (simpler than the form parameter, and simpler than
trying to allow repositioning), and could easily become the
defacto standard for systems that support it, if it were
permitted.

| Incidentally, I think Ted's history is totally wrong here.
| Append is seriously useful functionality for Sequential and Text_IO. Its
| provision in Stream_IO seems to have been mostly a matter of (possibly
| misguided) orthogonality.
| At least that is the way that Dewar remembers history!

Yes, the addition to Stream_IO was clearly last-minute, and for
orthogonality.
The inclusion for Sequential and Text_IO is what I was taking about,
where providing a mapping for O_APPEND is the historical basis.

--Ted

From dewar@gnat.com Fri Apr 10 12:20 EST 1998
Return-Path: <dewar@gnat.com>
Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4)
	id MAA20861; Fri, 10 Apr 1998 12:20:07 GMT
Received: by nile.gnat.com (5.0/1.20)
	id AA22760; Fri, 10 Apr 98 08:17:54 EDT
Date: Fri, 10 Apr 98 08:17:54 EDT
From: dewar@gnat.com (Robert Dewar)
Message-Id: <9804101217.AA22760@nile.gnat.com>
To: baker@dad.cs.fsu.edu, dewar@gnat.com
Subject: Re: AI95-00085
Content-Type: text
Content-Length: 5379
Status: R

<<I'd like to see this be the required semantics, for UNIX systems,
e.g. by the POSIX Ada bindings.
>>

Ah! But that is a TOTALLY different matter. Nothing we are talking about
here requires any compiler to actually *support* what you want.

If you want to go far beyond the AI, appropriate in the context of the POSIX
Ada bindings, but clearly not in the context of the AI, and make such a
requirement, then that's fine, but a Form parameter is a perfectly good
(actually preferable) path to implementing this requirement.

<<Moreover, last time we revised POSIX.5, people argued that we did not
need this (complicated and unwieldy) form parameter business any more,
given Ada'95 support for Append mode.
>>

Too bad that people did not know what they were talking about! Clealry
the Ada semantics as defined at the time did not support this argument.
This is not an unknown issue, as my comments quoted from the GNAT sources
show, I was aware of this issue years ago when I first implemented append
mode. Indeed it seemed quite obvious to me that the RM was inconsitent
with the use of the Unix append mode. I found it annoying, but there are
many things an implementor finds annoying with the language! Actually I
found it much more annoying that you don't know whether to open a file
in read or update mode when you first open it.

<<Does GNAT support such a form parameter?
>>

Certainly not! As I mentioned to you earlier, no customer of ours has ever
indicated that this was of interest.

<<The inclusion for Sequential and Text_IO is what I was taking about,
where providing a mapping for O_APPEND is the historical basis.
>>

No, that's wrong. The desire for append here has nothing necessarily to do
with synchronization with multiple processes. It is a much simpler
requirement, namely to be able to write starting at the end of the file.
Recall that neither Sequential_IO nor Text_IO allow the possibility of
positioning, so this fundamental capability, needed on all operating
systems, and available on all operating systems, was missing from Ada 83.
The revision requests in this area mentioned nothing about O_APPEND or the
need for providing this unix/posix special synchronization mechanism.

Indeed, NONE of the Ada 95 discussions raised the issue of this syncrhonization
question. Perhaps it was always in the minds of the Posix folks who were
involved in the discussion, and perhaps they always *knew* that this was
the crucial issue, but if so, it is too bad that they (a) did not speak up
and (b) let the language be designed in a manner inconsistent with the
use of Unix append without comment.

I have been involved in all the Ada 95 discussions, but not the Posix
discussions. However, I would comment that I always thought that the
basic motivation for the Posix append was the more general requirement.
After all, the main use of the append mode in Unix is for the simple
general requirement of doing appends. Even in Unix, the synchronization
lockout use is somewhat specialized (at least that is my experience in
seeing the mode used in real programs).

By the way, what does the ANSI C standard have to say about appending
exactly, can someone provide the quote. I think compatibility with C,
as opposed to compatibility with Unix, always provides a stronger
form of the compatibility argument.

I certainly have no objection to the Ada/Posix interface requiring support
of the Posix/Unix type append. However

*even if* the AI is written to allow the default behavior to mirror
the Posix way of doing append, I would be *really annoyed* if the Posix
binding required you to change your compiler to implement things this
way.

Ted says

<<In contrast to the form parameter, O_APPEND is very simple to
implement (simpler than the form parameter, and simpler than
trying to allow repositioning), and could easily become the
defacto standard for systems that support it, if it were
permitted.
>>

That is completely incorrect. Implementing append by changing the current
semantics, which our customers have been using for four years is not at
all trivial. We will have to do a survey to see who would be affected,
and most probably implement some complex mechanism to provide back
compatibility to those who are affected.

Implementing a form parameter is a simple job which should take only a day
or two, and be completely upwards compatible.

When the ARG changes the language in a manner that requires existing
compilers to change their implementation, and existing users to modify
their code, they are behaving in the most aggressive manner possible.
This level of aggression is only approrpriate for absolutely major
issues (we did even dare to do this for the 256 character case -- have
indeed we EVER done this).

I do not begin to believe that this issue is of sufficient importance
to warrant a requirement that existing implementations change, and I
strongly recommend that the POSIX group meet their (very reasonable)
requirement by other means than requiring language and implementation
chanes.

Yes, we might have done things differently 5 years ago if the issue had
been clearly understood, but since no one brought it to the table then,
it is too late now.

Ada 95 is not under construction! We are talking about features that have
been available to users for four years, following a standard that has been
available for over two years.

From baker Fri Apr 10 08:05:59 1998
To: dewar@gnat.com
Subject: RE: AI95-00085
Content-Length: 600

| I never saw an explicit request for Append mode for Stream_IO ...
| Ted, can you give the exact reference, I dug a little, but could not
| find anything at all.

I'm sorry I don't have the original (foot-think) revision requests
any more, due to lack of storage space.  If you have the original
revision requests documents on-line, and can't find it there, then
I may have to partially concede that I remember it wrong.  Of
course my memory is weighted toward what was said at the
requirements workshops (e.g. Soderfors and Ft. Walton Beach), and
that is not written down anywhere to check.

--Ted

From dewar@gnat.com Fri Apr 10 12:29 EST 1998
Return-Path: <dewar@gnat.com>
Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4)
	id MAA20879; Fri, 10 Apr 1998 12:29:48 GMT
Received: by nile.gnat.com (5.0/1.20)
	id AA22843; Fri, 10 Apr 98 08:27:35 EDT
Date: Fri, 10 Apr 98 08:27:35 EDT
From: dewar@gnat.com (Robert Dewar)
Message-Id: <9804101227.AA22843@nile.gnat.com>
To: baker@dad.cs.fsu.edu, dewar@gnat.com
Subject: RE: AI95-00085
Content-Type: text
Content-Length: 3415
Status: R

<<I'm sorry I don't have the original (foot-think) revision requests
any more, due to lack of storage space.  If you have the original
revision requests documents on-line, and can't find it there, then
I may have to partially concede that I remember it wrong.  Of
course my memory is weighted toward what was said at the
requirements workshops (e.g. Soderfors and Ft. Walton Beach), and
that is not written down anywhere to check.
>>

I was at Soderfors and at Ft. Walton Beach, and I am 99% sure that the
syncrhonization issue was never mentioned.

I think what happened is that people just *assumed* that the Ada append
mode would map into the Unix append mode, and so it was not discussed.

I must say I made this same assumption. It was a surprise to find out
that this was not the case, but I only noticed this later on when I
came to implement it.

Interestingly, in GNAT, we do not even use append mode for the Text_IO
and Sequential_IO cases, because of orthogonality in the implementation
for the Stream_IO case. That's probably undesirable, but actually until
this discussion, I was completely unaware of the synchronization issue,
and of course it is completely outside the Ada 95 semantics.

Indeed, relying on Append_File mapping into the Unix mode append even
for Text_IO or Sequential_IO is incorrect implementation dependent
coding in any case, so it is not clear that a user should rely on it.

In the GNAT documentation (I am not talking the sources now, but the user
level documentation), things are quite clear:

Table 6-1 Open and Create Call Modes

                               OPEN           CREATE
Append_File                    "r+"             "w+"
In_File                        "r"              "w+"
Out_File (Direct_IO)           "r+"             "w"
Out_File (all other cases)     "w"              "w"
Inout_File                     "r+"             "w+"

If text file translation is required, then either b or t
is added to the mode, depending on the setting of Text. Text file
translation refers to the mapping of CR/LF sequences in an external file
to LF characters internally. This mapping only occurs in DOS and
DOS-like systems, and is not relevant to other systems.

A special case occurs with Stream_IO. As shown in the above table, the
file is initially opened in r or w mode for the
In_File and Out_File cases. If a Set_Mode operation
subsequently requires switching from reading to writing or vice-versa,
then the file is reopened in r+ mode to permit the required operation.

I think it is quite important to provide this kind of documentation. Note
that it is quite clear from this that we do NOT use the a or a+ modes for
Append_File, and as I said before, no user has ever questioned this
decision. I can't say all our users read the documentation carefully,
but I can say that a lot do.

GNAT does provide a mechanism for opening a Text_IO or Sequential_IO
file using a stream value obtained from a C library call (or C code
making such a library call), and so this is the well defined mechanism
in GNAT for making use of the Unix append modes (you will then of course
get an exception if you try to do a positioning operation). I would assume
that any of our customers needing the special synchronization mechanisms
of Unix/Posix are using this approach, which seems quite appropriate, since
it nicely encapsulates the implementation dependent part of the process.

From baker Fri Apr 10 09:02:32 1998
To: dewar@gnat.com
Subject: Re: AI95-00085
Content-Length: 771

| That is completely incorrect. Implementing append by changing the current
| semantics, which our customers have been using for four years is not at
| all trivial. We will have to do a survey to see who would be affected,
| and most probably implement some complex mechanism to provide back
| compatibility to those who are affected.
...
| Ada 95 is not under construction! We are talking about features that have
| been available to users for four years, following a standard that has been
| available for over two years.

I may have lost the thread by now, but I thought the AI came
because an implementation was raising Use_Error in this situation,
and that I've been trying to support the position that such an
implementation be allowed to continue to do so.

--Ted

From baker Fri Apr 10 09:08:54 1998
To: dewar@gnat.com
Subject: RE: AI95-00085
Content-Length: 610

| this discussion, I was completely unaware of the synchronization issue,
| and of course it is completely outside the Ada 95 semantics.

It is not exactly synchronization (between processes), but
atomicity.  All individual I/O operations are atomic up to some
level of granularity defined by the OS (usually one disk block).
The special issue here is atomicity of the combined seek-to-end +
write.  With append mode, you do get something you cannot do with
the combination of lseek and write.

In this regard, the UNIX kind of append mode also provides
functionality beyond Ada's Direct_IO interfaces.

--Ted

From dewar@gnat.com Fri Apr 10 12:53 EST 1998
Return-Path: <dewar@gnat.com>
Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4)
	id MAA20939; Fri, 10 Apr 1998 12:53:39 GMT
Received: by nile.gnat.com (5.0/1.20)
	id AA23135; Fri, 10 Apr 98 08:51:27 EDT
Date: Fri, 10 Apr 98 08:51:27 EDT
From: dewar@gnat.com (Robert Dewar)
Message-Id: <9804101251.AA23135@nile.gnat.com>
To: baker@dad.cs.fsu.edu
Subject: RE: AI95-00085
Content-Type: text
Content-Length: 338
Status: R

Ted, is it really the case that my last few messages on this went only to
you. If so that's really aggravating, since the whole point is to discuss
this with the ARG.

GRRRR! I see that is the case. Have you saved these messages, if so, PLEASE
forward them to the whole group. It is really a waste of time for just the
two of us to talk.

From baker Fri Apr 10 09:12:14 1998
To: dewar@gnat.com
Subject: Re: previous mail
Content-Length: 213

I'll find the mail, and forward it.  I hope it does not overwhelm
everybody.  I was getting uncomfortable about the volume and length
of it, hence my limiting my reply to you only, a couple of cycles
back.  --Ted

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Friday, April 10, 1998 8:43 AM
Subject: 	Re: last message of Baker/Dewar exchange

| <<I may have lost the thread by now, but I thought the AI came
| because an implementation was raising Use_Error in this situation,
| and that I've been trying to support the position that such an
| implementation be allowed to continue to do so.
| >>

| Indeed, there is an implementation that wants to negate the standard by
| raising Use_Error for all uses of positioning when Append_File is used.

| We object because it is perfectly possible, even on Unix, and even if you
| use the Unix append mode, to support positioning (by reopening the file).

My disagreement here is that if you do this you hide a significant
semantic event that the user did not expect (i.e., closing and
then reopening) and which may have unwanted side-effects.

If there were some important special Ada functionality here, this
might be worth while, but no one has given any argument yet that
explains how this functionality is needed.  First of all, if
repositioning is needed (even in other modes) with Stream_IO, why
can we do without it for Text_IO and Sequential_IO?  Second, if
the semantics of repositioning in Append mode are the same as a
combination of other operations that a user could have done, why
do we need this mode at all?

| But the suggestion that ALL implementations be required to raise Use_Error
| (your suggestion) is way out of line.

That is *not* what I'm suggesting.  I'm suggesting that implementations
be allowed (by the ARG) to raise Use_Error.

What may have misled you is that I've also argued that other
standards groups, like POSIX, be allowed (by the ARG) to narrow
the semantic window for a particular OS.

--Ted

****************************************************************


From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Friday, April 10, 1998 9:20 AM
To: 	Arg95@sw-eng.falls-church.va.us; baker@dad.cs.fsu.edu
Subject: 	Re: last message of Baker/Dewar exchange

<<My disagreement here is that if you do this you hide a significant
semantic event that the user did not expect (i.e., closing and
then reopening) and which may have unwanted side-effects.
>>

But these implicit reopens are part of life anyway, you have no way of
avoiding these for the switch from reading to writing in a stream file,
you certainly don't want to open all files in update mode just in case
you may want to write, since that would mean you could not use stream_io
on read only files.

<<If there were some important special Ada functionality here, this
might be worth while, but no one has given any argument yet that
explains how this functionality is needed.  First of all, if
repositioning is needed (even in other modes) with Stream_IO, why
can we do without it for Text_IO and Sequential_IO?  Second, if
the semantics of repositioning in Append mode are the same as a
combination of other operations that a user could have done, why
do we need this mode at all?
>>

It is VERY useful to be able to reposition for stream files, code does
this ALL the time. Sequential_IO and Text_IO are at a higher level where
repositioning makes zero sense.


<<| But the suggestion that ALL implementations be required to raise Use_Error
| (your suggestion) is way out of line.

That is *not* what I'm suggesting.  I'm suggesting that implementations
be allowed (by the ARG) to raise Use_Error.
>>


No, you are clearly suggesting this, *so that* you can then have the Posix
binding require its use, so your ultimate agenda is precisely to require
implementations to raise Use_Error. If this is not your intent (in the long
run, including what you want the Posix interface to do), please clarify.

To me, introducing implementation dependence here is a bad thing. Both in
the case of direct Ada use of Stream_IO and the POSIX interface (which
should support O_APPEND), I see no advantages in introducing implementation
dependence.

****************************************************************

From: 	David Emery[SMTP:emery@mitre.org]
Sent: 	Friday, April 10, 1998 9:00 AM
Subject: 	Re: last message of Baker/Dewar exchange

Well, since my name was mentioned, I guess I'm entitled to contribute $.02 :-)

Ted's recollection about 'intent' for Append_Mode matches my own.  This was
discussed within the POSIX.5 group, within the SIGAda ALIWG, and in some
emails
I remember back and forth during the Mapping and Revision process.

The particular "desire" was to provide access to the functionality provided
by Unix's
O_APPEND for Ada files.  There were two motivations for this:
	1.  Conformity with Unix semantics
	2.  It's generally useful.
Within Unix, the 'formal semantics' mention atomic writes to the end of the
file.  At the
user level, this degree of specification wasn't mentioned to my
recollection.  But the
"semantics by analogy" from users was clear:  We want Ada's Append_Mode to act
like Unix's O_APPEND.

I don't have my copy of POSIX.1 with me (I'm now working nearly full-time
in the
Pentagon :-(, so I can't look at the POSIX-specified for O_APPEND and
lseek().

Robert Dewar argues, rightly, for surveying the vendors to see the degree
of implementation
approaches. At the same time, it is appropriate in an issue like this to
survey users, too.
Thus a message to comp.lang.ada and/or TeamAda that asks people about what
they
expect when they use Append_Mode would seem to me to be appropriate.

Personally, I've never thought about the interactions between Append_Mode and
file positioning, because my expected use for Append_Mode has always been for
"write-only" files (e.g. logfiles), where file seeking makes no
application-level sense.

				dave

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Friday, April 10, 1998 9:31 AM
Subject: 	Re: last message of Baker/Dewar exchange

<<Robert Dewar argues, rightly, for surveying the vendors to see the degree
of implementation
approaches. At the same time, it is appropriate in an issue like this to
survey users, too.
Thus a message to comp.lang.ada and/or TeamAda that asks people about what
they
expect when they use Append_Mode would seem to me to be appropriate.

>>

We can get any answer we like here:


What do you think Append_File should do? Most people will simply answer
that it should start writing at the end of the file.

Do you think Append_File should map to the Unix append mode? Most people
will simply answer yes.


<<Personally, I've never thought about the interactions between Append_Mode and
file positioning, because my expected use for Append_Mode has always been for
"write-only" files (e.g. logfiles), where file seeking makes no
application-level sense.
>>

So for you, the obvious implementation which Ted rejects would be fine.

The issue is the following.


Suppose you open a file in append mode and write some stuff

Then suppose you do a positioning operation.


Ted thinks it is fine to reject all positioning operations.

Robert thinks it is easy enough to implement them, so you should.

Ted doesn't like Robert's implementation because it involves closing and
opening the file, he prefers to be able to completely forbid the positioning.

The trouble is that the RM absolutely clearly and unambiguously, and
intentionally, allows positioning and append_file mode to be combined.


What Ted (and Dave perhaps) are recommending is that implementations be free
to say

"I could implement that, in a manner consistent with Ada semantics, but I
don't think it's a good idea, and I would never use it, so I am not going
to implement it."

That's a heavy burden to support!

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Friday, April 10, 1998 9:34 AM
Subject: 	Re: last message of Baker/Dewar exchange

<<I noticed just now that I forgot to add the other main problem,
i.e., that if you reposition you are no longer appending.  In effect,
you have changed modes.
>>

But appending only had to do with the initial position, as Ada is defined.
Yes, you are certainly not appending in the Unix sense, but you know that,
because the Unix sense of appending would not have allowed you to position.


Actually I think one sense in which positioning and append are perfectly
compatible from a conceptual point of view is if you are allowed to position,
but only at or after the original append point. I hoped to be able to read
this permission into the RM, since it seems to make a lot more sense, but
in fact I concluded that it was absolutely clear that the RM requires you
to be able to position behind the starting position, which I agree is bogus.

I really think it would be useful to know what other compilers are
currently doing, can we have some data on this please?

****************************************************************

From: 	Robert I. Eachus[SMTP:eachus@mitre.org]
Sent: 	Friday, April 10, 1998 10:40 AM
Subject: 	Re: AI95-00085

At 09:30 AM 4/10/98 -0400, Robert Dewar wrote:

> Indeed, there is an implementation that wants to negate the standard by
> raising Use_Error for all uses of positioning when Append_File is used.
>
> We object because it is perfectly possible, even on Unix, and even if you
> use the Unix append mode, to support positioning (by reopening the file).
>
> But the suggestion that ALL implementations be required to raise Use_Error
> (your suggestion) is way out of line.

    Can we all agree to this?  It allows both sides to go away happy.  Yes,
it does require a bit of extra work if the implementation chooses to make
use of O_APPEND.  And, yes, there are cases where the reopening can be
detected or will fail.  But it preserves the current RM semantics, and will
allow implementations (including GNAT) to use O_APPEND if they choose to.

    A little more on what happens with the reopen.  It is perfectly
reasonable for an implementation (IMHO) to raise Use_Error in some cases
(see A.12.1(33)) --repositioning on a pipe for example.  But in general, an
implementation must support repositioning on a stream connected to a "real"
disk file, no matter what mode it was originally opened in.  And there some
are anomalies that should be left as implementation defined.  (For example,
if the file is deleted by one task while a second task is writing to the
associated stream.)

                                        Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

****************************************************************

From: 	Pascal Leroy[SMTP:phl@Rational.Com]
Sent: 	Tuesday, April 14, 1998 6:15 AM
Subject: 	Re: copy of Baker/Dewar exchange

> By the way, what does the ANSI C standard have to say about appending
> exactly, can someone provide the quote. I think compatibility with C,
> as opposed to compatibility with Unix, always provides a stronger
> form of the compatibility argument.

The only interesting thing that I could find in the ANSI C RM is this:

<<7.9.7.3 The fputc function

If the file cannot support positioning requests, or if the stream was opened
with append mode, the character is appended to the output stream.>>

This is rather terse, but still much more precise than RM95 A.12.

To summarize the ANSI C semantics is as follows:

Positioning is supported for files opened in append mode (except of course for
some special files, in an implementation-dependent manner).  Positioning is in
fact _useful_ if the file is opened in a+ or ab+ mode (reading and appending)
because you can go back to some earlier location in the file, and start
reading from there.  However, each time you _write_ to the file, you go to the
end.

As already noted, it is rather unfortunate that Ada doesn't have read+write or
read+append modes.

If we wanted to mimic the C semantics, we would have to decide that Set_Index
moves the index (and does not raise an exception); and that the next Write
moves to the end of file no matter what.  (I am not saying that we should do
that, btw.)

Pascal

_____________________________________________________________________
Pascal Leroy                                    +33.1.30.12.09.68
pleroy@rational.com                             +33.1.30.12.09.66 FAX

****************************************************************

From: 	Robert Dewar[SMTP:dewar@gnat.com]
Sent: 	Tuesday, April 14, 1998 5:24 AM
Subject: 	Re: copy of Baker/Dewar exchange

So one interesting conclusion from Pascal's summary of the ANSI C semantics
is that forbidding all positioning for stream files in append mode, as
suggested by some, would actually *decrease* compatibility with C,
interesting ...

Pity no one looked at the ANSI C standard more carefully if people think
that compatibility with C is so important (I don't, my feeling is that
if you want compatibiliy with C, please use the C routines directly,
you can still us the stream attributes, so you don't lose any expressive
power).

****************************************************************

From: 	Ted Baker[SMTP:baker@dad.cs.fsu.edu]
Sent: 	Tuesday, April 14, 1998 7:53 AM
Subject: 	Re: copy of Baker/Dewar exchange

| So one interesting conclusion from Pascal's summary of the ANSI C semantics
| is that forbidding all positioning for stream files in append mode, as
| suggested by some, would actually *decrease* compatibility with C,

It would be compatible. Given that you can't read from an Ada file
in append mode, the only use of such files is to write.  In C
repositioning in append mode has no effect on subsequent writes
(they still go to the end of the file), you effectively cannot
reposition.

Of course, I agree that it would have been nicer if we had all
sensible combinations of read, write, append in Ada, but we don't.

| Pity no one looked at the ANSI C standard more carefully if people think
| that compatibility with C is so important (I don't, my feeling is that
| if you want compatibiliy with C, please use the C routines directly,
| you can still us the stream attributes, so you don't lose any expressive
| power).

Admittedly, the only way to get true C compabibility.

--Ted

****************************************************************

From: Randy Brukardt
Sent: Monday, January 15, 2001 5:41 PM

I have received 16 test results for the AI-85 test so far. From these
results, I can draw some preliminary conclusions. Then I'll go on to add my
take on the AI as a whole.

My first conclusion is that an ACATS test for Ada.Streams.Stream_IO is
desperately needed. The testing uncovered bugs in the implementation of
Stream_IO in 4 compilers, irrespective of the use of Append_File. Most of
the problems have to do with the maintenance of the file index.

Secondly, most compilers that target Unix do in fact use Append_File as
meaning O_APPEND. However, none of the compilers come anywhere near the RM
or in fact the current draft of the AI. Most compilers allow calls to
Set_Index, then essentially ignore that those happened. Some even go so far
as to return the Index value has having been changed by Set_Index, but then
write at a different location in the file.

----

Technical Corrigendum 1 rewrote A.12.1 to clearly define the handling of the
current file index in a positionable file to handle Defect Report 8652/0055
(AI-26). With those changes, the RM is much clearer on the handling of
stream files.

The current draft of AI-00085 is in fact a confirmation of the RM. Indeed,
everything it says about Append_File mode stream files follows directly from
the RM. Thus, in one sense, resolving this issue is easy.

I see two problems with a strict reading of the RM.

First, there is a hole in the RM in that it never says that Read and Write
without index parameters actually Read or Write at the current index.
(Compare A.12.1(30) to A.8.5(3).) Certainly, the intent of AI-26 was that
the model of positionable stream files is similar to Direct_IO files. It
certainly would be odd if that didn't include the location of Reading and
Writing. Probably a sentence like the following needs to be added to the RM:

    "For a file that supports positioning, Read without a Positive_Count
parameter starts reading at the current index, and Write without a
Positive_Count parameter starts writing at the current index."

I would assume the above is non-controversial, but since quite a few
compilers ignore it, perhaps I'm wrong.

Anyway, assuming the above, it is clear that a file opened with O_APPEND
cannot support positioning. That follows since a Set_Index followed by a
Write will not write at the current index, rather it will write at the end
of the file. This is essentially what the test tries to do.

This would cleanly provide the solution to the problem. However, (depending
on how the RM is read), it may not be able to be used for a disk file.

The problem is that A.12.1(2/1) clearly defines positioning as a property of
the external file. However, the mode of the file (which is what O_APPEND is)
is a property of the INTERNAL file. Thus, the mode of the file cannot affect
its positionability.

To stir up the pot some, this is the position that the test I distributed
takes. It essentially tries writing the file in Append_File mode, then read
the file in In_File mode. It then checks if the file is positionable. It
then closes the file, reopens it, and again checks positionability.

Note that this position essentially means that O_APPEND cannot be used to
open disk files, or that the mode has to be silently changed if a call to
Index or Set_Index is made. That's because any later (or earlier!) attempt
to use the file must have the same positionability; and everyone agrees that
we want positioning on Unix disk files opened in the In_File and Out_File
modes.

The question is how to handle this:
1) Do nothing, issue an ACATS test similar to the one I distributed. This
would force Unix based implementations to limit the use of O_APPEND, and
make its use semantically invisible. (Alternatively, they could provide
access to it via a form parameter.)
2) Change the RM to allow positioning to be a property of the internal file.
Then, an implementation could use O_APPEND for Append_File by making such
files not positionable. An ACATS test similar to the one I distributed would
be issued, with all of the "RLBCheck"s removed. (Append_File would only be
lightly tested in this case).
3) Reject my argument above. In that case, no conclusions can be drawn from
the RM on what files are and are not positionable. The effect and testing
are similar to (2).

Based on prior discussions, I'd expect that the number of people choosing
(1) (for portability) and (2-3) (for capability) would be roughly equal.
Perhaps some people have changed views?

Personally, I don't see the point of insisting on Append_File mode being
implemented with O_APPEND. The behavior of O_APPEND is rather unusual, and
only applicable to a single OS. I would rather see a portable version of
Stream_IO, with a form parameter to select the OS-specific behavior. This
also would be more compatible with existing practice: all compilers tested
allowed Index with Append_File; only one disallowed Set_Index with
Append_File.

If O_APPEND is deemed important, then clearly the file is not positionable,
and Index and Set_Index raise Use_Error. This is less compatible, both for
existing users (because they might be using Index), and for users porting
from one system to another (code that would work on Windows would fail on
Linux, for example). I think this sort of incompatibility needs a important
reason to be triggered, and O_APPEND doesn't provide it.

****************************************************************

From: Tucker Taft
Sent: Monday, January 15, 2001 8:08 PM

When we talk about the external file vs. the internal file, I believe
we are talking about properties external to the Ada program vs.
properties internal to the Ada program.  There is nothing that
says that a property of an external file can't change during its
lifetime, as a side effect of what the Ada program does.
Hence, if you use O_APPEND, then the external file is
not positionable, because the external environment (i.e. the operating
system) does not support positioning during appending.

Hence, I don't think you can argue that the implementation is
wrong if the positionability of the external file changes during
its lifetime.  For example, even though size is a property of
the external file, it can change frequently.

In fact, if something is a property of an external file, then
there is not much you can say about it or how it changes.
A.14 makes it clear that properties of an *internal* file are
not affected by operations performed on other internal files
associated with the same external file, but there is no real
limitation on what can affect the properties of an external file.

Given the above, I would say the positionability of a file
is not guaranteed to stay the same forever, though of course it
would be annoying if it changed capriciously, and presumably the
market would weed out such implementations.

The reason Unix provides the O_APPEND feature is that it is
very common for multiple processes to share a file in Unix (e.g.
a log file), and O_APPEND does the "right" thing in that environment.
Other operating systems tend to be much less "friendly" to allowing
multiple processes to have the same file open for writing, and hence
the need for an O_APPEND is less critical.

****************************************************************

From: Pascal Leroy
Sent: Tuesday, January 16, 2001 2:58 AM

> Personally, I don't see the point of insisting on Append_File mode being
> implemented with O_APPEND.

As Tuck pointed out, O_APPEND _is_ important to allow sharing of files to work
correctly (e.g. log files, pipes, etc).  Any change that would require an
implementation to stop using O_APPEND would be an inacceptable incompatibility,
since there is real code out there that depends on this capability.

The alternative is to disallow positioning on an append file, and while it is
an incompatibility too, it seems much less severe.

> The behavior of O_APPEND is rather unusual, and
> only applicable to a single OS.

It is certainly not true that O_APPEND is "applicable to a single OS".  I don't
have an ANSI C manual handy, but open(2) is part of the standard libraries, and
I wouldn't be surprised if O_APPEND were part of the standard.  At any rate, OS
vendors want C code to port easily to their OS, and so they better have some
support for O_APPEND.

Case in point: look for O_APPEND in the Visual C++ documentation and you'll
find that it exists with exactly the same semantics as on Unix:

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 7:46 AM

<<> Personally, I don't see the point of insisting on Append_File mode being
> implemented with O_APPEND.>>

It seems obvious that with the current language definition, Append_File
cannot be implemented with O_APPEND. The following is from the GNAT sources:

   --  Note: we do not use "a" or "a+" for Append_File, since this would not
   --  work in the case of stream files, where even if in append file mode,
   --  you can reset to earlier points in the file. The caller must use the
   --  Append_Set routine to deal with the necessary positioning.

Yes, that's unfortunate, but the design clearly does not permit this
behavior. To "fix" this certainly requires a language change, and
it seems very late to be making such a change to me.

****************************************************************

From: Pascal Leroy
Sent: Tuesday, January 16, 2001 8:00 AM

> Yes, that's unfortunate, but the design clearly does not permit this
> behavior. To "fix" this certainly requires a language change, and
> it seems very late to be making such a change to me.

Well, either way there is going to be some amount of incompatibility.  Randy's
experiments have proven that there are a number of compilers out there which do
use O_APPEND, and we cannot ignore this fact.  We could act as language
lawyers, invent semantics for positioning on append files, and disregard the
fact that there are probably users which depend on the O_APPEND behavior and
hose code is going to break.

However, that seems very wrong to me.  At this late stage, we should only add
incompatibilities if they buy us something.  Sorry, I cannot get excited about
positioning on append files, I am not even sure what this is supposed to mean
in the first place.

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 2:04 PM

Why do you think invention is needed here? THe RM seems clear to me. Once
again, I think the only way the RM is puzzling is if you assume that
append = O_APPEND.

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 2:01 PM

<<However, that seems very wrong to me.  At this late stage, we should only add
incompatibilities if they buy us something.  Sorry, I cannot get excited about
positioning on append files, I am not even sure what this is supposed to mean
in the first place.>>

I see no problem in the current definition, EXCEPT that it prevents the
use of O_APPEND for append mode, but nothing in the RM says that you have
to provide this capability. Since O_APPEND is inconsistent with Ada semantics,
it should be controlled by a form parameter of some kind, we can certainly
agree on the form parameter.

I have not yet heard any specifics on how GNAT is not conforming to the RM,
so that needs some report and investigation, but for sure I would assume
that any compiler using O_APPEND cannot possibly be conforming.

Furthermore, we can't make a non-backwards compatible change at this
stage.

Now of course if all compilers are significantly broken with respect to
the current RM (note "all" and "significantly") then we have more
flexibility in what we can do :-)

****************************************************************

From: Tucker Taft
Sent: Tuesday, January 16, 2001 2:50 PM

Robert Dewar wrote:
> Now of course if all compilers are significantly broken with respect to
> the current RM (note "all" and "significantly") then we have more
> flexibility in what we can do :-)

I would agree that GNAT is conforming if it avoids O_APPEND and
allows meaningful positioning in Append_File mode.
However, you have never answered my point that it is likewise
permissible for an implementation to make positionability
change during the lifetime of a file.  Of course, with GNAT
going one way and Rational (and some others) going another way,
we don't provide the commonality we are seeking.  But I think
either is a legitimate interpretation of the RM, since there
is some much flexibility in how implementations handle "external"
file characteristics.

The next question is whether implementors are motivated to eliminate
this implementation-dependence.  I am hearing that Pascal and you
don't agree about what interpretation is best, so that it might
make sense to simply deem it implementation-defined whether
Stream_IO files are positionable while in Append_File mode,
thereby warning users of the possible non-portability.

****************************************************************

From: Ada Rapporteur Group of WG9; managing the Ada issues [ARG@acm.org]
on behalf of Robert Dewar [dewar@GNAT.COM]
Sent: Tuesday, January 16, 2001 2:55 PM
To: ARG@acm.org
Subject: Re: AI-85 analysis and discussion.

<<However, you have never answered my point that it is likewise
permissible for an implementation to make positionability
change during the lifetime of a file.>>

I don't understand this point, which is why I did not answer it.

To me the RM requires that ordinary files in append mode be positionable.

****************************************************************
From: Robert Dewar
Sent: Tuesday, January 16, 2001 2:56 PM

I guess what surprises me here is that the issue has come up, it seemed
obvious to me that you cannot use O_APPEND for append mode, and I never
considered doing so.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, January 16, 2001 4:18 PM

My position is very simply:

O_APPEND is an operating system specific facility. We have a mechanism for
accessing operating system facilities in I/O: the Form parameter. Ergo, O_APPEND
should be controlled by a form parameter.

OTOH, I can't really understand Robert's objection. Robert said: "To me the RM
requires that ordinary files in append mode be positionable." I don't see much
in the RM that would support that position.

The RM says (the second paragraph was added by the Corrigendum):

1   [The subprograms in the child package Streams.Stream_IO provide control over
stream files. Access to a stream file is either sequential, via a call on Read
or Write to transfer an array of stream elements, or positional (if supported by
the implementation for the given file), by specifying a relative index for an
element. Since a stream file can be converted to a Stream_Access value, calling
stream-oriented attribute subprograms of different element types with the same
Stream_Access value provides heterogeneous input-output.] See 13.13 for a
general discussion of streams.

Static Semantics

1.1/1
       {8652/0055} The elements of a stream file are stream elements. If
       positioning is supported for the specified external file, a current index
       and current size are maintained for the file as described in A.8. If
       positioning is not supported, a current index is not maintained, and the
       current size is implementation defined.


Nothing in this text makes any requirement on an implementation as to whether a
particular file is positionable. Indeed, which files are positionable isn't even
required to be documented, as it isn't even marked as implementation-defined.
(Is that an oversight in the RM or was it really intended to be unspecified?)

Thus, an implementation can make anything it wants non-positionable, as long as
it properly implements the semantics of a non-positionable file. (Most
implementations implementing O_APPEND do not do this currently, but there has
been general agreement that this would be an acceptable resolution of the
issue.)

The Corrigendum text, however, does make it clear that positioning is a
property of the external file (not the internal file).

I argued that therefore whether or not a file is positionable cannot depend on
the file mode (which clearly is a property of the internal file). Tucker
counter-argued that the properties of an external file are those external to
the Ada program, and that there is no requirement that they cannot change
during the life of the file.

If my argument is correct, then a implementation which treated a file opened
with Append_File mode as non-positionable could not later treat the same file
as positionable. This would be testable, and would effectively prevent using
O_APPEND mode to open files (because you could not know when you were going to
later open a file with another mode).

If Tucker's argument is correct, then there is no problem with allowing the
same file to later be opened (or reset to) another mode, and then allowing
positioning in that mode.

I have to admit, I find my own argument weak. It seems dubious to argue about
requirements on unspecified stuff.

However, if you reject that argument, then there is no problem whatsoever
(vis-a-vis the RM) with opening files with O_APPEND, as long as such files are
treated as not being positionable. In this case:
    -- The AI is a confirmation (because everything needed is already in the
       RM);
    -- An ACATS test to check that Append_File mode files are either
       positionable (and then positioning had better work right) or
       non-positionable should be issued, but this would have no effect on
       whether O_APPEND is used in opening files.

We could consider adding some implementation advice on this issue. For instance,
we could say that it is morally wrong to treat an ordinary disk file as
non-positionable. (I certainly agree with that statement.) But from prior
discussion, I think it is clear that we would never get consensus on such
implementation advice. So I think it would be a waste of time to bother
discussing that; I'd rather work on Tucker's Interfaces proposal :-).

****************************************************************

From: Randy Brukardt
Sent: Tuesday, January 16, 2001 4:33 PM

> The next question is whether implementors are motivated to eliminate
> this implementation-dependence.  I am hearing that Pascal and you
> don't agree about what interpretation is best, so that it might
> make sense to simply deem it implementation-defined whether
> Stream_IO files are positionable while in Append_File mode,
> thereby warning users of the possible non-portability.

This is a horrible idea. That would let us argue about this for many more
years, and for what?

Whether files are positionable in the first place is unspecified. Perhaps that
should have been implementation-defined. To me, the only viable change to the
RM would be to add a statement that "Which files support positioning, if any,
is implementation-defined." That has nothing to do with Append_File; it just
fixes an oversight in the RM. (I'd suggest fixing the Read without Index
parameter goof as well, if we're making wording changes to this clause.)
Let's not even think about getting append mode mixed up in it!!!!

I think Implementation Advice saying "O_APPEND considered harmful." would also
be appropriate, but I have no illusions that any such thing would ever pass.

****************************************************************

From: Tucker Taft
Sent: Tuesday, January 16, 2001 4:34 PM

Robert Dewar wrote:
>
> <<However, you have never answered my point that it is likewise
> permissible for an implementation to make positionability
> change during the lifetime of a file.  Of course, with GNAT
> >>
>
> I don't understand this point, which is why I did not answer it.
>
> To me the RM requires that ordinary files in append mode be positionable.

The RM doesn't specify under what circumstances "positioning
is supported for the [external] file."  It says what should
happen if positioning is or is not supported, but it doesn't
say when or if positioning should be supported.  So it seems
permissible for positioning to be supported some of the time,
and not at other times.  In particular, it seems permissible
for positioning to be disallowed while the file is being
appended-to.

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 5:03 PM

I disagree, if you allow this sort of thing, you legitimize any
kind of bizarre behavior, and that is not the intention of the
design. The only way positioning should not be supported here is
if there is no reasonable implementation model. But there IS a
reasonable implementation model for implementing append *as described
in the RM*. If it was the intention of the design to allow O_APPEND
as an implementation model here, then the design is indeed messed up.

But we certainly have a reasonable implemenation model that provides the
semanitcs in the RM. Therfore there is no good basis from an RM point of
view of permitting an alternative implementation model that does not work.

By the way, I agree it is *horrible* that the design disallows the use
of O_APPEND, but that's the way the design is. You do not solve design
problems by allowing implementations to arbitrarily ignore the semantic
requirements.

It is VERY late to be addressing this issue, certainly we can't disallow
positioning of stream files. I guess at this stage, if we really want to
allow the O_APPEND implementations to be considered conforming, we just
bite the bullet and say that it is implementation defined whether
positioning is allowed for stream files in append mode -- seems pretty
ugly, but I see no alternative.

We just *can't* go down Tuck's path here, it legitimizes absolutely any
possible malfunction in the implementation of I/O. For example suppose
we run

  Print_Line ("Hello");

and the output is

  Hello!

and I note to the world that I have chosen an implementation model which
does not support lines not ending in exclamation marks so I supply one
by default (if you won't let me do that I will have to raise Use_Error).

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 5:09 PM

<<Whether files are positionable in the first place is unspecified. Perhaps
that should have been implementation-defined. To me, the only viable change
to the RM would be to add a statement that "Which files support positioning,
if any, is implementation-defined." That has nothing to do with Append_File;
it just fixes an oversight in the RM. (I'd suggest fixing the Read without
Index parameter goof as well, if we're making wording changes to this
clause.) Let's not even think about getting append mode mixed up in it!!!!
>>

I object to this, the only valid reason for not supporting positioning
in the general case (forget about append) is if the underlying system
does not support positioning. It is plain wrong for an Ada runtime to
prevent positioning an arbitrary file.

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 5:13 PM

<<Nothing in this text makes any requirement on an implementation as to
whether a particular file is positionable. Indeed, which files are
positionable isn't even required to be documented, as it isn't even marked
as implementation-defined. (Is that an oversight in the RM or was it really
intended to be unspecified?)>>

I think this is perfectly deliberate AND reasonable.

All files are positionable, there is no hint that this is not the case.

On certain implementations, it may be the case that [stand by for RM
1.1.3(6) quote :-)] it is "impossible or impratical to avoid [the
failure to support positioning] given the implementation's execution
environment" for certain files, in which case, it is fine not to
support it.

But 1.1.3(6) is THE only justification for not supporting positioning,
and weakening this any further is a very undesirable precedent.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, January 16, 2001 5:31 PM
To: ARG@ACM.ORG
Subject: Re: AI-85 analysis and discussion.

The words "(if supported by the implementation for the given file)"
certainly imply to me that it is perfectly reasonable for a file to be
non-positionable. If the intent was that all files must be positionable
unless there is an insurmountable reason for them not to be, I would expect
the wording to be very different.

I certainly think it is a good idea for implementations to try to make as
many files positionable as possible, but I certainly don't see any
requirement to do so.

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 5:36 PM

(if supported by the implementation for the given file)

So it boils down to whether you think this is talking about whether the
underlying file system supports this, or whether the implementor feels
like doing it or not. For example, a liberal reading would allow an
implementor to say that no files mentioned in the ACVC tests support
positioning, but any other file does :-)

(I would not approve this interpretation)

****************************************************************

From: Randy Brukardt
Sent: Tuesday, January 16, 2001 5:50 PM

Honestly, I don't see a lot of difference. Tucker's argument is that the
underlying file system does not support positioning for a file opened in
O_APPEND, and certainly that is true. I personally think that using O_APPEND is
a bad design decision by the implementor, but I don't think that the RM can do
much to prevent bad design decisions. Certainly the RM would not object to an
implementation that used 256-bit float numbers to do all integer calculations,
but I certainly hope no one actually implements Ada that way.

****************************************************************

From: Robert Dewar
Sent: Tuesday, January 16, 2001 9:08 PM

<<Honestly, I don't see a lot of difference. Tucker's argument is that the
underlying file system does not support positioning for a file opened in
O_APPEND, and certainly that is true. I personally think that using O_APPEND
is a bad design decision by the implementor, but I don't think that the RM
can do much to prevent bad design decisions. Certainly the RM would not
object to an implementation that used 256-bit float numbers to do all
integer calculations, but I certainly hope no one actually implements Ada
that way.
>>

Bogus analogy!

The choice of O_APPEND is not semantically neutral. The choice of
256-bit float for integer IS semantically neutral (omitting timing,
which we never include in such considerations)(.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, January 16, 2001 5:44 PM

> But 1.1.3(6) is THE only justification for not supporting positioning,
> and weakening this any further is a very undesirable precedent.

The more I think about this, the more bizarre it seems to me. I don't know of
any (other) place in the RM where it goes to substantial lengths to define what
happens in a case that could only happen via a 1.1.3(6) argument. The usual
idea is that such arguments are used to allow deviations from the standard.
But in this case, we have a clearly defined concept (a file which does not
support positioning). Such a concept would be necessary only if the standard
anticipated a need for such files. But if such files are expected to exist,
the standard certainly wouldn't then require all files to be positionable.
(Or if it did intend to, it certainly would say so!)

Now everyone knows that A.12.1 was horribly incomplete (just look at all of the
stuff we had to add in the Corrigendum, and even then we didn't fix all of the
problems). So the notion that whatever was meant to be said about supports
positioning vs. does not support positioning was omitted is easy to believe.
But I find your interpretation to be the least likely possible meaning.

****************************************************************

From: Pascal Leroy
Sent: Wednesday, January 17, 2001 3:26 AM

> Why do you think invention is needed here? THe RM seems clear to me. Once
> again, I think the only way the RM is puzzling is if you assume that
> append = O_APPEND.

I'm sorry, but ignoring Ada for a moment, in general computing terms, I don't
understand what the interaction between appending and positioning is supposed
to be.

A.7(11) describes the various file modes and says "these values correspond to
the cases where only reading, only writing, or only appending are to be
performed."  Note the phrase "only appending."  To me, the verb append implies
that the new data are added at the end.  I am reading the following definition
from the American Heritage Dictionary: "to add as a supplement or an appendix."

So, if you are positioned somewhere in the middle of the file and you write,
what is supposed to happen?  Either you write where you are (and in this case
do you truncate? do you overwrite?) and this is not "only appending." Or you
go to the end and write from there (which is the O_APPEND semantics) and
positioning is vacuous.

Call it invention or call it clarification if you prefer, I believe we have to
do something to explain the interaction between appending and positioning.

> But we certainly have a reasonable implementation model that provides the
> semantics in the RM. Therefore there is no good basis from an RM point of
> view of permitting an alternative implementation model that does not work.

Just out of curiosity, when "appending in the middle" of a file, do you
truncate or do you overwrite?  Both seem reasonable implementation models to me,
and the RM doesn't appear to have a preference.

> By the way, I agree it is *horrible* that the design disallows the use
> of O_APPEND, but that's the way the design is. You do not solve design
> problems by allowing implementations to arbitrarily ignore the semantic
> requirements.

The design is badly broken, in the sense that it mixes two abstractions that
have nothing to do with each other.  One is the append-only file, e.g. a
socket, a pipe, a log file, etc.  And the other is the
file-as-an-array-of-bytes, with positioning/addressing capability.  In
isolations, these abstractions make perfect sense.  But we have a problem when
we try to divine what positioning/addressing means for an append-only file.
So if the semantic requirements don't make any sense, well, ignoring them is
probably a very reasonable option...

> It is VERY late to be addressing this issue...

I hate to mention this, but one of the reasons it is very late is that this AI
has been dormant since October '98 because one Robert Dewar didn't do his
homework ;-)

> ... I guess at this stage, if we really want to
> allow the O_APPEND implementations to be considered conforming, we just
> bite the bullet and say that it is implementation defined whether
> positioning is allowed for stream files in append mode -- seems pretty
> ugly, but I see no alternative.

I can live with that, and I believe it's close to Tuck's proposal.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 8:07 AM

<<A.7(11) describes the various file modes and says "these values correspond to
the cases where only reading, only writing, or only appending are to be
performed."  Note the phrase "only appending."  To me, the verb append implies
that the new data are added at the end.  I am reading the following definition
from the American Heritage Dictionary: "to add as a supplement or an
appendix.">>

I don't see the relevance of the dictionary definition. The notion of
appending is something defined by the RM, so we can only get information
on what it means from there. If append was replaced by zork in the RM,
this would not chnage the meaning of the concept.

<<So, if you are positioned somewhere in the middle of the file and you write,
what is supposed to happen?  Either you write where you are (and in this case
you truncate? do you overwrite?) and this is not "only appending." Or you go t
the end and write from there (which is the O_APPEND semantics) and positioning
is vacuous.>>

You overwrite, I don't see any expectation for truncation. Where in the RM
do you get any implication that truncation would occur (it seems to me that
your only reason for thinking this is reading semantics into the english
word append that are not in the RM).

It is a useful excercise here to indeed replace append with zork, and
then reread the RM. I think if that is done, then things get clearer.

I think it is clear that the truncation model is wrong, since A.8.2(9)
pretty clearly says that the truncation model applies ONLY to
sequential files.

Pascal's view of append would correspond to compeltely disallowing
positioning, as a matter not of implementation restriction, but from
a fundamental logical point of view -- positioning incomaptible with
appending, but the RM clearly allows positioning on append files, of
that there is no doubt. This proves that the notion that Pascal has
of append is not what the RM is talking about.

There are actually three models that make conceptual sense:

1. Append has only to do with where you position the file at the end on
open and reset, and otherwise is the same as Out (really should be In Out,
but that's another story, the failure of stream io to have in out is to
me also an annoying glitch).

In this model, positioning is fully allowed, and you can position to a
point BEFORE the original append point.

2. Append gives you a view of a file that consists only of the part
you are adding, where you can position, but not before the original
append point. To me that would make a whole lot more sense than 1,
but there is no hint in the RM that positioning is restricted in this
way, or that it is reasonable to restrict it.

3. Append does not allow positioning at all. Again, there is no hint in
the RM that this is the intended meaning.

All three models are well defined. I can only read the RM as permitting
the first model, which to me is the least desirable of the three, I agree,
so it would be fine to me to devise commonly accepted form parameters to
specify either of the other two actions.

One interesting data point here is that in GNAT we never use O_APPEND, not
even for sequential files, because it is easier to treat all file types
the same, and no one ever noticed, let alone asked for this to be changed,
so this issue, though once on my radar screen when I first implemented
things, since I was concerned about the consequences of the RM definition,
long ago fell off the radar screen.

What is the motivation for considering this as having higher priority now?
Has some user actually run into problems, or is this just a matter of the
old concern finally surfacing after years of neglect :-)

****************************************************************

From: Pascal Leroy
Sent: Wednesday, January 17, 2001 8:16 AM

> I don't see the relevance of the dictionary definition. The notion of
> appending is something defined by the RM, so we can only get information
> on what it means from there. If append was replaced by zork in the RM,
> this would not chnage the meaning of the concept.

The word "append" is _not_ defined by the RM.  Could you please tell me where
you find such a definition?  That word certainly doesn't appear in the index.
(Well, the word Append, with a capital, appears in the index, but only because
it's the name of some unrelated string subprograms.)

When a term is not defined by the RM, 1.3(1) refers us to the Webster's Third
New International Dictionary of the English Language.  I don't happen to have
this dictionary, but I draw two conclusions from this: (1) a dictionary
definition is certainly relevant here and (2) your "zork" argument is
irrelevant unless you can point to a definition of "zork" in the RM.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 8:39 AM

<<The word "append" is _not_ defined by the RM.  Could you please tell me where
you find such a definition?  That word certainly doesn't appear in the index.
(Well, the word Append, with a capital, appears in the index, but only because
it's the name of some unrelated string subprograms.)>>

THat's the point, it is an undefined term, replace it with Zork, and then
you can only derive information from what's actually said about it, which
is that on open and reset, the file is positioned at the end, there is NO
other information, therefore the term has no other implication.

The only reason you are building in other implications is that you think
you know that the english word append should correspond to what the same
english word happens to mean in some operating system.

There is no implication in the RM that append has ANY effect on the
ability to position a file, or that it has any effect other than
affecting the initial position after an open or reset.

The one clue that all is not right in the world is the "only appending"
statement (let's remember that in the version we are examining, it says
"only zorking").

Now if zorking has only to do with initial positioning, it is indeed a bit
odd to consider that when you do subsequent writes, you are zorking (i.e.
that the definition of zorking in this context is "writing after initially
positioning the file at the end of the file, but intermediate positioning
operations can have occurred).

But following this thought would definitely lead to an expectation of
a definite prohibition against positioning for append files, and this
prohibition just is not there currently.

****************************************************************

From: Ada Rapporteur Group of WG9; managing the Ada issues [ARG@ACM.ORG]
on behalf of Pascal Leroy [pleroy@rational.com]
Sent: Wednesday, January 17, 2001 8:56 AM
To: ARG@ACM.ORG
Subject: Re: AI-85 analysis and discussion.

> THat's the point, it is an undefined term, replace it with Zork, and then
> you can only derive information from what's actually said about it, which
> is that on open and reset, the file is positioned at the end, there is NO
> other information, therefore the term has no other implication.

Hmm, using the same reasoning, I will point out that the word "end" is nowhere
defined in the RM, and that you should really interpret the wording as saying
that when a file is opened in zork mode, it is positioned at the zark after Open
and Reset.  Now of course you seem to be reading too much in the word "end",
because you know what that term means in English, but there is no implication
that "end" in the RM (which as we know is really zark) means the end of the
file.

Other than the fact that an RM where all undefined terms would be replaced by
some z---rk word would be interesting reading, I find your reasoning absurd.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 9:12 AM

<<Other than the fact that an RM where all undefined terms would be replaced
by some z---rk word would be interesting reading, I find your reasoning
absurd.>>

Well perhaps you find it absurd, but this is precisely the way that the
RM is written.

Common english words like end are taken to have their normal meaning,

but any technical term like append is quite different. To read into
the term append a requirement that is nowhere in the RM is simply not
valid in my view.

I ask you once again, where in the RM do you find any hint or implication
that append never allows positioning?

If there was such an implication, why would it not be stated?

To me, when I read the RM, it was immediately apparent that it was a little
odd that append allowed general positioning.

Actually, my concern was that I thought it made no sense to position BEHIND
the append point, positioning after the append point makes perfect sense
to me (and is consistent with your dictionary viewpoint, in the following
sense, that opening a file in append mode, writing some stuff, with
possible positioning as you go, and then closing it, results in appending
stuff to the file in the dictionary sense).

Indeed if you go with the dictionary definition, it definitely corresponds
to this definition, and not to the UNIX viewpoint of not allowing positioning
at all.

Think of the following:

a. please append your views to the end of the chapter.

b. ok (starts scribbling, after a while, corrects a missing cross on a t
in what was scribbled and continues)

c. Hey! I told you to append your views, what the %$#@$%#@ are you doing
going back and changing something you wrote

b. Huh?

--------------
Now, on the other hand, why does the implication that you can position
BEFORE the append point worry me (that's the question I originally brought
up), because now the dialog is:

a. please append ....

b. OK (starts scribbling, after a while goes and changes spelling in the
original material)

c. HEY! I told you to append your comments, not to go changing things

d. Oh, sorry ...


--------------

So I really don't think the dictionary definition justifies the no
positioning viewpoint at all.

Indeed it seems clear to me that if you want a no positioning regime, you
should use a special FORM parameter, this is something different entirely
than what the RM has in mind.

The dictionary definition is sometimes useful in figuring out what someone
had in mind, but it does not affect the definition in a case like this.

You are doing something different here, you are not trying to connect
append in the RM to append in the dictionary, but rather append in the RM
to O_APPEND in Unix. There is simply no justification for this attempted
connection in RM terms.

****************************************************************

From: Pascal Leroy
Sent: Wednesday, January 17, 2001 9:25 AM

> You are doing something different here, you are not trying to connect
> append in the RM to append in the dictionary, but rather append in the RM
> to O_APPEND in Unix. There is simply no justification for this attempted
> connection in RM terms.

No, that's an unfair comment.  I am not trying to connect anything to O_APPEND.
I am trying to understand what is the meaning of positioning for an append
file, based on what the RM says and based on the usual meaning of English
words.  This seems to be a prerequisite to taking any decision on this issue.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 9:44 AM

OK, here is my position.

1. Based on what the RM says, it seems clear that general positioning is
allowed, because it is not disallowed.

2. Based on the normal english meaning of the word, appending means adding
something to the file, and excludes modifications to the existing part of
the file. This means that positioning for modification should be restricted
to the appended section.

I can't see any justification, EITHER in the RM OR in the dictionary for
forbidding positioning completely.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 10:06 AM

Robert Dewar wrote:
> ... I guess at this stage, if we really want to
> allow the O_APPEND implementations to be considered conforming, we just
> bite the bullet and say that it is implementation defined whether
> positioning is allowed for stream files in append mode -- seems pretty
> ugly, but I see no alternative.

We should be careful here to allow what to me is the most natural
expectation, namely that positioning is allowed, but the position is
relative to the part you are adding.

****************************************************************

From: Tucker Taft
Sent: Wednesday, January 17, 2001 10:02 AM

Robert Dewar wrote:
> ... I guess at this stage, if we really want to
> allow the O_APPEND implementations to be considered conforming, we just
> bite the bullet and say that it is implementation defined whether
> positioning is allowed for stream files in append mode -- seems pretty
> ugly, but I see no alternative.

This is the compromise that Pascal and I are also supporting.

I believe there are ample "legalisms" to justify our position,
but I also hate the idea of implementors generally using legalisms
to support all kinds of bizarre behavior, so I see Robert's point as well.

I have some memories of why we added Append_File mode as related
to being able to use Unix O_APPEND, because it has nice properties in
the presence of file sharing.  Otherwise, why go to the trouble of
having Append_File mode, since you can just do a Set_Index to the
size of the file + 1?

In any case, at this point I recommend we adopt the compromise
suggested above by Robert, and give up on the more philosophical
discussion.

****************************************************************

From: Randy Brukardt
Sent: Wednesday, January 17, 2001 2:15 PM

Tuck said:

> I have some memories of why we added Append_File mode as related
> to being able to use Unix O_APPEND, because it has nice properties in
> the presence of file sharing.  Otherwise, why go to the trouble of
> having Append_File mode, since you can just do a Set_Index to the
> size of the file + 1?

If the language design really had done this, I would have fought it
tooth-and-nail.

But the reason we have Append_File mode is so that Sequential_IO and Text_IO
files can be appended on. They don't have an alternative way to get the end,
and this was a significant problem. I certainly did not know about the odd
semantics of the Unix Append mode, and I certainly would not have supported
anything that would include that semantics in the Ada standard.

Why we have an Append_File on stream files I do not know. I think it was
misguided orthogonality. (It's clear that no one reviewed the Stream_IO section
carefully, given how unclear and incomplete it is.) The only possible use for
such a mode would be on a file that does not support positioning.

(Note that I always thought that the semantics of Append_File were what Robert
describes as option 1: "Append has only to do with where you position the file
at the end on open and reset, and otherwise is the same as Out." I'm quite
amazed that anyone would think otherwise -- which I think is the reason this
debate is so hard to resolve. Since the RM doesn't say much about Append_File
mode, everyone seems to think it matches whatever their expectation is; and
those are all different.)

Back to Tuck:
> In any case, at this point I recommend we adopt the compromise
> suggested above by Robert, and give up on the more philosophical
> discussion.

I don't have a problem with the compromise, except that it appears to be
completely unnecessary. The only real question is whether the RM requires files
to be positionable. If the answer is no (which I believe is the only reasonable
position), then the RM as it stands is flexible enough to handle any problems.
If the answer is yes (Robert's position), then we need not only Robert's
compromise, but also some statement to the effect that files are supposed to
support positioning unless impossible or impractical. There is no such
implication currently in the RM.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 7:13 PM

<<Why we have an Append_File on stream files I do not know. I think it was
misguided orthogonality. (It's clear that no one reviewed the Stream_IO
section carefully, given how unclear and incomplete it is.) The only
possible use for such a mode would be on a file that does not support
positioning.>>

Ah well, we have QUITE different reactions then, because to me it is
clear that Append is very useful, and SHOULD allow positioning, but
should NOT allow positioning before the append point. That's what
I would expect from *my* dictionary definition :-)

****************************************************************

From: Randy Brukardt
Sent: Wednesday, January 17, 2001 8:12 PM

Yes, that is becoming clear to me.

Of your previously mentioned three possible semantics, I think #1 was meant,
you think #2 was meant, and Pascal (based on his previous mail) seems to
think #3 was meant. This explains why it is so hard to get agreement on this
issue!

It's unfortunate that neither the RM nor AARM give any inkling as to what the
designers of Ada 95 had in mind. Thus we all guess, and apparently we guess
differently, based on our experiences.

While it might be valuable to tie this down, I suspect that reaching consensus
on it could be very difficult. I simply don't think it is worth the time, and
given that the RM doesn't seem to outright prohibit what some implementors
want, I think I would prefer to simply say nothing (beyond noting in the AI
that some implementations may choose to make Append_File mode files
non-positionable in order to use O_APPEND). This has the advantage of making
the AI a confirmation, and would allow us to test it soon (based on my recent
tests, I think that would benefit Ada users by improving the likelihood that
Stream_IO is implemented correctly).

****************************************************************

From: Randy Brukardt
Sent: Wednesday, January 17, 2001 8:27 PM

To make it clear how I propose to resolve this AI:

I would leave the AI as a confirmation. I would add some text to the discussion
section to make the resolution clear, that is that we're adopted Robert's
compromise.

That is, "It is implementation defined whether positioning is allowed for
stream files in append mode." Saying so in the discussion would be intended
to put up a flag for users. I'd also figure to add that as an annotation to
the AARM the next time it is revised.

I do agree that the AI should mention it. I do not agree that the RM needs
wording changes to mention this; it follows from the fact that it is
unspecified whether positioning is allowed for a particular stream file.

The RM does need a wording change to say the obvious: that Read (and Write)
without an index parameter read (or write) at the current index if the file
supports positioning. I don't think we need to clutter this AI (which also
contains three other questions, all confirmations) with that wording.

Personally, I think that the RM should also have said that "It is
implementation defined whether positioning is allowed for a particular stream
file." (rather than leaving it unspecified), so that users can verify that
their implementations are not playing games. It is unfortunate that this was
left unspecified in the RM, because it does open the door to all kinds of
games. But I think that the market is better at eliminating these sorts of
games rather than the RM and ARG and ACATS.

If we want to consider either of these wording changes, I think it would be
best to give them their own AI (separate from the other questions of AI-85).
And I think that AI should get a fairly low priority.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 7:20 PM

<<I agree that technically, an implementation need not support
positioning on any file, but we are also concerned about what
we consider "normal" and "common" behavior, so users have some
reasonable expectation.>>

I disagree, the ACATS tests should test positioning on normal files,
and any implementation not providing this behavior should be considered
to fail these tests.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 10:05 PM

I must say I cannot understand Randy's thinking that interpretation one
was meant (though of course, I agree that's what is said). How can
you possibly regard a sequence of actions that rewrites an existing
file as being doing "only append" operations.

****************************************************************

From: Randy Brukardt
Sent: Wednesday, January 17, 2001 10:26 PM

> I must say I cannot understand Randy's thinking that interpretation one
> was meant (though of course, I agree that's what is said). How can
> you possibly regard a sequence of actions that rewrites an existing
> file as being doing "only append" operations.

I wasn't aware of the statement Pascal dug up until this morning (A.7(11)).
My understanding of Append_File goes back a lot further than that.

Besides, I do not believe that there is any value to an "only appending"
mode for stream files (because, like you, I believe virtually all such files
should be positionable). So I view that wording as yet another unfortunate
oversight in a pile of them for Stream_IO.

Finally, if you take that literally, then positioning on Append_File mode
files must be banned. (That is, interpretation #3.) I don't think we can
afford to take that position: not a single compiler tested to date actually
treated Append_File mode files as not supporting positioning. Insisting on
that now would be a massive incompatibility.

****************************************************************

From: Jean-Pierre Rosen
Sent: Thursday, January 18, 2001 11:09 AM

> > I must say I cannot understand Randy's thinking that interpretation one
> > was meant (though of course, I agree that's what is said). How can
> > you possibly regard a sequence of actions that rewrites an existing
> > file as being doing "only append" operations.
>
> I wasn't aware of the statement Pascal dug up until this morning (A.7(11)).
> My understanding of Append_File goes back a lot further than that.
>
Following this thread, I reread that paragraph. Nobody mentionned it, so I just
add as a piece of information that direct files do *not* have an append mode.
This supports the idea that direct positionning and Append_Mode are somehow
incompatible.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 10:06 PM

<<That is, "It is implementation defined whether positioning is allowed for
stream files in append mode." Saying so in the discussion would be intended
to put up a flag for users. I'd also figure to add that as an annotation to
the AARM the next time it is revised.>>

Let me repeat again that this wording is NOT acceptable, since it implies
only intepretations 1 and 3, and does not permit 2.

****************************************************************

From: Randy Brukardt
Sent: Wednesday, January 17, 2001 10:20 PM

I can't imagine any (simple) wording which would allow 2. Doing so would
require an entire new set of semantics for Index/Set_Index etc. Such a
"partially positionable" file would certainly need changes in a bunch of
paragraphs of the standard. I don't think this problem is anywhere important
enough to justify any language change at all, much less a wholesale one.

****************************************************************

From: Robert Dewar
Sent: Wednesday, January 17, 2001 11:20 PM

Maybe not, maybe the external file is only the appended part in this
case, after all the RM does not define what a file is.

This actually seems a perfectly valid semantics.

****************************************************************

From: Ted Baker
Sent: Thursday, January 18, 2001 6:48 AM

| <<That is, "It is implementation defined whether positioning is allowed for
| stream files in append mode." Saying so in the discussion would be intended
| to put up a flag for users. I'd also figure to add that as an annotation to
| the AARM the next time it is revised.
| >> -- Randy

| Let me repeat again that this wording is NOT acceptable, since it implies
| only intepretations 1 and 3, and does not permit 2.  --Robert

It is an essential feature of append mode that the file CANNOT be
repositioned, or to be more precise, it cannot stay repositioned
if repositions, since every write operation is required to append to
the end of the file.

That is an invariant on which applications will rely.

****************************************************************

From: Robert Dewar
Sent: Thursday, January 18, 2001 8:08 AM

<<It is an essential feature of append mode that the file CANNOT be
repositioned, or to be more precise, it cannot stay repositioned
if repositions, since every write operation is required to append to
the end of the file.>>

Not append mode as defined in the RM. Please justify such sweeping
statements by reference to the RM.

<<That is an invariant on which applications will rely.>>

Not if they are reading the RM!

****************************************************************

From: Ted Baker
Sent: Thursday, January 18, 2001 5:25 AM

| > I have some memories of why we added Append_File mode as related
| > to being able to use Unix O_APPEND, because it has nice properties in
| > the presence of file sharing.  Otherwise, why go to the trouble of
| > having Append_File mode, since you can just do a Set_Index to the
| > size of the file + 1?

| If the language design really had done this, I would have fought it
| tooth-and-nail.
| ... I always thought that the semantics of Append_File were what
| Robert describes as option 1: "Append has only to do with where you position
| the file at the end on open and reset, and otherwise is the same as Out."
| I'm quite amazed that anyone would think otherwise ...

Sorry, Randy, but Tucker is right.  I was there, and remember
being one of the ones who made the argument, remember that this
view was voiced, clearly.  It may be that you and some others
heard what you wanted to hear, though.

The members of the POSIX Ada bindings working group brought this
up.  Unix/POSIX systems require that for this mode "the file
offset shall be set to the end of the file prior to EACH write".

On the "better" implementations, this is done atomically, so that
multiple processes may log to the same file concurrently, without
fear of losing any data due to overwriting.  However, there seems
to be some controversy within the Unix standards group over
whether atomicity is required, since the above-quoted phrase not
clearly state that.

I hope you will realize that atomicity of writing and
positioning to the end here offers a distinct benefit, just
as does atomicity of checking for prior existence of a file
and creating a new file of a given name.

Because of these benefits, these features are provided in atomic
form on most modern operating systems.  Our goal in getting
an append mode into Ada 95 was to not deny access to these
valuable features to Ada language users.

****************************************************************

From: Robert Dewar
Sent: Thursday, January 18, 2001 8:02 AM

<<The members of the POSIX Ada bindings working group brought this
up.  Unix/POSIX systems require that for this mode "the file
offset shall be set to the end of the file prior to EACH write".>>

Maybe, but there is no trace of this requirement in the RM.

<<I hope you will realize that atomicity of writing and
positioning to the end here offers a distinct benefit, just
as does atomicity of checking for prior existence of a file
and creating a new file of a given name.>>

Maybe, but there is no trace of this feature in the RM

<<Because of these benefits, these features are provided in atomic
form on most modern operating systems.  Our goal in getting
an append mode into Ada 95 was to not deny access to these
valuable features to Ada language users.>>

Maybe, but then "you" [whoever the you is] messed up. In particular,
consider the following

        10   Severs the association between the given file and its associated
        external file.  The given file is left closed.  In addition, for
        sequential files, if the file being closed has mode Out_File or
        Append_File, then the last element written since the most recent open
        or reset is the last element that can be read from the file.  If no

why are stream files excluded here?

I do not contest that it would be useful to have the mode you suggest,
I just do not see that there is any trace of intent to have this mode
for *STREAM* files in the RM, though of course sequential files do
indeed work the way you mention (is your memory of what happened perhaps
related to sequential files). I say this because IF there was ever any
clear discussion of the intent here, then what is in the RM is even
more decisive, because it means it was written with a clear awareness
of the issue.

So far, at least some people in this discussion have been assuming that
the RM is "wrong" in this area as a result of NOT properly considering
the issue of stream files and append. If indeed as Ted remembers, it was
carefully considered, then one would be more inclined to think that the
design in the RM is indeed deliberate, and in particular that the
exclusion of stream files in para 10 above was deliberate.

****************************************************************

From: Pascal Leroy
Sent: Thursday, January 18, 2001 8:33 AM

> Maybe, but then "you" [whoever the you is] messed up. In particular,
> consider the following
>
>         10   Severs the association between the given file and its associated
>         external file.  The given file is left closed.  In addition, for
>         sequential files, if the file being closed has mode Out_File or
>         Append_File, then the last element written since the most recent open
>         or reset is the last element that can be read from the file.  If no
>
> why are stream files excluded here?

It is interesting to note that AARM A.8(2.a-b) explains that the authors of the
RM considered defining 'sequential files' to mean Sequential_IO and Stream_IO
files.  There is a possibility that the omission of stream files from the above
paragraph is a historical accident resulting from a shifting definition of
'sequential file'.  Of course, there is no way to know at this point, and any
reasoning based on reconstructing the intent is weak.  However, we must not
ignore the fact that the text of the RM as we read it today might merely be the
result of incomplete/incorrect editing.

****************************************************************

From: Robert Dewar
Sent: Thursday, January 18, 2001 8:44 AM

<<It is interesting to note that AARM A.8(2.a-b) explains that the authors of th
RM considered defining 'sequential files' to mean Sequential_IO and Stream_IO
files.  There is a possibility that the omission of stream files from the abov
paragraph is a historical accident resulting from a shifting definition of
'sequential file'.  Of course, there is no way to know at this point, and any
reasoning based on reconstructing the intent is weak.  However, we must not
ignore the fact that the text of the RM as we read it today might merely be th
result of incomplete/incorrect editing.>>

An interesting thought indeed.

Reacting to Ted's message, if he insists that applications will rely
on the meaning of append as he sees it, and if (as we know) some but
not other implementations provide the semantics he thinks applications
will rely on, then we have a significant mess on our hands.

What we don't know is the following:

1. For implementations that use O_APPEND and do not allow positioning
(Rational, ....) How many applications really are relying on this
semantics.

2. For implementations that allow positioning (GNAT, ...) how many
applications actually do the positioning operations in append mode.

Now let's assume both these sets are non-empty. Clearly applications
in category 1 are worrisome because a) they will be blown up in
non-repeatable non-obvious ways by either an ARG ruling requiring
positioning, or moving to a positioning compiler. Furthermore, if
such a change was made by the ARG, then the fix (switch to sequential
io) is painful (and may not even work on compilers not supporting the
necessary file sharing semantics).

If applications actually DO positioning operations in append mode,
then if the positioning were disallowed, the situation is not so bad.

a) they get a clear use error with a message saying what is going on

b) the fix is simply to reset into write mode before the positioning
operation (note that for implementations allowing my first model of
append, the one Randy thinks is most natural, and the one I think that
the RM specifies now, append mode is equivalent in any case to just
opening in write mode and positioning to the end manually).

This makes it clear that we really cannot "fix" this situation by
requiring model 1 semantics, even if Robert and Randy are quite sure
that's what the RM says. That is, pragmatically, if the set of users
who are depending on O_APPEND is non-zero (and O_APPEND compilers have
no way of knowing if this is a non-empty set), then implementors of
O_APPEND compilers are simply NOT going to change their implementations
just to satisfy some ARG ruling that does not affect any of their
current customers except negatively.

On the other hand, non O_APPEND compilers can likely change to O_APPEND
mode with less disruption, so that's not out of the question to mandate.

If Ted is really right, and there are people who are depending on this,
even if they can't justify such action from the RM, then maybe it is a
mistake to leave this implementation dependent. At the very least I
would suggest no action, rather than making it implementation dependent,
since a specific action of making it ID does not improve the situation
in my view.

****************************************************************

From: Ted Baker
Sent: Thursday, January 18, 2001 4:27 PM

| 1. For implementations that use O_APPEND and do not allow positioning
| (Rational, ....) How many applications really are relying on this
| semantics.

As noted in my later e-mail, the POSIX semantics seem to allow
nominal repositioning, so that one might interrogate the position
(or do a read) and find that the position is not at the end of the
file, but that would have no effect on the output.  It would still
atomically reposition to the end of the file each time a write is
done.  I don't recall ever hearing that anybody relied on this,
though.  I believe the main use of O_APPEND is for safe logging.

****************************************************************

From: Ted Baker
Sent: Thursday, January 18, 2001 4:12 PM

| ... is your memory of what happened perhaps
| related to sequential files ..

Possibly.  In the POSIX context we were talking about operations
on "files", not streams.  I believe the original user input asking
for an append mode dated back into the Ada 83 days and was based
on the idea of adding a new mode to the Ada 83 I/O model.  Streams
were not around in Ada yet.  My belief is that the two ideas met
later in the Ada 9X draft, without a lot of discussion of the
interactions.

BTW, it is interesting to note that the subject of O_APPEND has
led to interesting interpretation discussions in the POSIX domain
as well.  The question came up in the Austin Group discussions
about what happens when one repositions a file that was opened
with O_APPEND, e.g., using lseek().  The answer seems to be that
you can change the "current position" between writes (i.e.,
between appends), but when you do a write that position has no
effect (i.e., the writes still append to the end of the file).

--Ted

Excerpts from Austin Group e-mail on this subject:

        ... the requirements stated for open()
(with or without the O_APPEND flag) is that the file offset for the new
file descriptor is set to be 0 (see XSH6d4, P1351, L27378-27379).  The
file offset doesn't move to the end of the file until one of the
functions that explicitly moves the file offset or writes data to that
file descriptor completes successfully.
        There is no requirement that the file offset always be set to
the end of file on a file opened with O_APPEND; only that all writes
add data to the end of the file rather than overwriting data already
present in the file.

****************************************************************

From: Randy Brukardt
Sent: Thursday, January 18, 2001 4:07 PM

> | If the language design really had done this, I would have fought it
> | tooth-and-nail.
> | ... I always thought that the semantics of Append_File were what
> | Robert describes as option 1: "Append has only to do with where you
> | position > | the file at the end on open and reset, and otherwise is
> | the same as Out."
> | I'm quite amazed that anyone would think otherwise ...
>
> Sorry, Randy, but Tucker is right.  I was there, and remember
> being one of the ones who made the argument, remember that this
> view was voiced, clearly.  It may be that you and some others
> heard what you wanted to hear, though.

I'm afraid you failed to read my statement carefully. I specifically said
"If the LANGUAGE DESIGN had done this...". I was well aware that some
POSIX-centric lobbying existed for Append_File, and I did my best to prevent
any OS-specific features from getting added to Ada 95. My view of append is
the only one that could make sense on all of the weird OSes to which Ada has
been ported (from DOS to POSIX, from VMS to OS-1100), and I made sure that
the RM language did not require anything beyond that view. If there had been
such language, I would have objected. (Of course, like everyone else, I
never gave Stream_IO a careful reading...)

> On the "better" implementations, this is done atomically, so that
> multiple processes may log to the same file concurrently, without
> fear of losing any data due to overwriting.  However, there seems
> to be some controversy within the Unix standards group over
> whether atomicity is required, since the above-quoted phrase not
> clearly state that.
>
> I hope you will realize that atomicity of writing and
> positioning to the end here offers a distinct benefit, just
> as does atomicity of checking for prior existence of a file
> and creating a new file of a given name.

There probably is a benefit of atomicity for low-level writing. But that
benefit does not extend to Stream_IO. That's because there is no limit on
the number of calls to the stream Write that a Write or Output attribute can
make (with the possible exception of an elementary type attribute). An Ada
implementation would have to go to heroic measures (*) in order to guarantee
that, and there certainly is no requirement in the RM to do so. Thus, any
program that depended on such atomicity is completely implementation and
target dependent.

For instance, consider two Ada programs, both using O_APPEND to write to a
log file. Each log entry is a record including a string message, a user id,
and an time stamp. The canonical semantics for 'Write would involve a call
to the stream Write for each component, including each individual character
of the message! An compiler could use fewer calls than this, but that is
only possible in isolated circumstances (no user defined attributes, no
representation clauses) and in any event is not guaranteed. If both programs
started writing a log item at the same time, the OS atomicity would only
guarantee that a particular call to Stream_IO.Write did not overlap another.
Thus, it is very likely that the two records would end up interspersed, and
they could not be read by any means at all.

The same problems could happen for Text_IO, but there the result is much
less likely to be fatal.

Thus, I conclude that atomicity of O_APPEND is not useful for Stream_IO
files (at least, not unless there are major changes to other parts of the
stream system). The other feature of O_APPEND (automatic seeking to the end)
is actually harmful for an stream file, as it increases the chance that what
is written cannot be read back. Thus, any effort to support O_APPEND on
stream files is wasted. (This argument may not apply to Text_IO files.)

                        Randy.


(*) A implementation that did want to guarantee atomicity of a particular
Write would have to do the following:
-- Define additional atomicity operations in the private part of
Root_Stream_Type, with null implementations;
-- Override these operations in the private part of Ada.Streams.Stream_IO;
-- Generate calls to these operations at the start and end of each top-level
stream attribute call;
-- Include a buffer and lock in each file object;
-- For a start_atomic, initialize a buffer, and lock a lock;
-- For each following Write, check that the correct task is doing the
operation (if not block it until end_atomic); write the contents into the
buffer.
-- For end_atomic, write the contents of the buffer to the OS and unlock the
lock.

This requires the ability to buffer an unknown number of stream elements per
operation, and thus requires some sort of dynamic allocation of buffers. The
lock could be omitted if the implementation is willing to make concurrent
access from the same partition erroneous (it seems very strange to insist on
atomicity between *different* partitions, but not care for the *same*
partition).

Even with this scheme, a user-defined stream could not take advantage of
these facilities (because they're not part of the public interface of
Root_Stream_Type). Thus, users would be denied the opportunity to use this
information in their own streams. (Claw, for instance, declares a stream
type in order to be able to allow arbitrary Ada objects to be written to the
Windows Clipboard. The Clipboard requires all of the data to be presented at
once, thus we need a buffering stream much like the one described above.)

Clearly, the RM does not require all of this buffering and extra stream
calls, and any programmer depending on its existence is
implementation-dependent in the worst way.

****************************************************************

From: Randy Brukardt
Sent: Thursday, January 18, 2001 4:22 PM

Robert wrote:

> What we don't know is the following:
>
> 1. For implementations that use O_APPEND and do not allow positioning
> (Rational, ....) How many applications really are relying on this
> semantics.
>
> 2. For implementations that allow positioning (GNAT, ...) how many
> applications actually do the positioning operations in append mode.

This is not a correct view of the actual situation. I realize that I haven't
distributed the test results yet, because results are still trickling in.

1. For implementations that use O_APPEND and do not allow positioning
   How many applications really are relying on this semantics.

The OcSystems compiler is the only (fielded) one that disallows the use of
Set_Index on a Append_File mode stream file. Tucker changed his
implementation during the early discussion, but I doubt any users have seen
that yet.

2. For implementations that allow positioning (GNAT, ...) how many
applications actually do the positioning operations in append mode.

OK, but there is a third possibility:

3. For implementations that use O_APPEND and allow positioning (but it still
writes at the end) (Rational, Green Hills, probably Aonix). How many
applications really are relying on this semantics. How many applications use
Append, rely on the seek to end semantics and also rely on Index (which
would be disallowed if positioning is).

Note that the last set might be empty, as the Index in some of the compilers
at least returns the value of Index as if the Ada semantics were followed,
as opposed to the correct value (where you will write).

> Now let's assume both these sets are non-empty. Clearly applications
> in category 1 are worrisome because a) they will be blown up in
> non-repeatable non-obvious ways by either an ARG ruling requiring
> positioning, or moving to a positioning compiler. Furthermore, if
> such a change was made by the ARG, then the fix (switch to sequential
> io) is painful (and may not even work on compilers not supporting the
> necessary file sharing semantics).

I think that the set of applications successfully in category 1 or 3a is
probably null, because of the reasons mentioned in my response to Ted. OTOH,
there may be applications in these categories that *think* they work, so the
impact may be non-zero.

> If applications actually DO positioning operations in append mode,
> then if the positioning were disallowed, the situation is not so bad.
>
> a) they get a clear use error with a message saying what is going on

Given the quality of exception handling in many Ada compilers, I would not
depend on the message part.

> b) the fix is simply to reset into write mode before the positioning
> operation (note that for implementations allowing my first model of
> append, the one Randy thinks is most natural, and the one I think that
> the RM specifies now, append mode is equivalent in any case to just
> opening in write mode and positioning to the end manually).

True enough. Append_File mode should never have been added to stream files
in the first place: it adds nothing, and it is dangerous to boot (if people
think that they can do POSIX O_APPEND with it, which almost never will
work).

****************************************************************

From: Robert Dewar
Sent: Thursday, January 18, 2001 8:52 PM

Randy you missed my point completely I am afraid, I was wondering how
many applications depend on O_APPEND semantics, sorry if I was not clear.
The critical thing about O_APPEND is the safety.

****************************************************************

From: Randy Brukardt
Sent: Thursday, January 18, 2001 10:05 PM

Humm, perhaps you missed my point, which is that for a Stream_IO file, there
almost never *is* any additional safety from using O_APPEND. That's because
a typical stream write is made up of many tiny writes, and another process
writing the same file would still jumble it up. It would only work if no
other process did write the file; but then you don't need O_APPEND
semantics!

Sequential files don't have this problem; it really only occurs with stream
files. (For a text log, it possibly could happen depending on how Text_IO is
designed, but it's unlikely to be fatal.)

The only time O_APPEND could be useful would be when the file consists
*only* of elementary types *and* the file needs to be positioned (presumably
in another mode). This seems unlikely enough to be the empty set. (Log files
are typically text files, and wouldn't be written with Stream_IO in the
first place.)

****************************************************************

From: Pascal Leroy
Sent: Friday, January 19, 2001 2:32 AM

Your reasoning is correct if you use T'Write or T'Output to write to a
Stream_IO file.

But on the other hand, it is perfectly possible to write to a Stream_IO file
using only Stream_IO.Write (perhaps because you do conversion to byte arrays
yourself), in which case the atomicity property of O_APPEND is applicable.

****************************************************************

From: Robert Dewar [dewar@gnat.com]
Sent: Friday, January 19, 2001 6:36 AM

Obviously you do NOT use "many tiny writes" if you are using Stream_IO
with O_APPEND and want indivisible semantics. You use one write of a
stream element array of appropriate length.

****************************************************************

From: Robert Dewar
Sent: Friday, January 19, 2001 6:39 AM

<<The only time O_APPEND could be useful would be when the file consists
*only* of elementary types *and* the file needs to be positioned (presumably
in another mode). This seems unlikely enough to be the empty set. (Log files
are typically text files, and wouldn't be written with Stream_IO in the
first place.)>>

This seems a REALLY strange claim. My version of the RM has:

   procedure Write
     (File : in File_Type;
      Item : in Stream_Element_Array;
      To   : in Positive_Count);

And allows arbitrarily long Item's to be written. yes, of course you have to
avoid doing individual little writes to the Stream of the file if you want
O_APPEND semantics for logging. But that's easy to program.

Now of course if people naively DO use the Stream, yes, then the whole
O_APPEND argument is junk.

Furthermore, if you are going to use the Write above, you could in practice
equally well use O_APPEND mode for sequential IO instantiated for
Stream_Element_Array (although that would put extra "green word" junk
into the file, which might be unacceptable).

Still, I agree, this makes it much less likely that there really IS anyone
depending on O_APPEND semantics out there after all. So I guess the argument
for switching to allowing O_APPEND semantics is much weaker.

****************************************************************

From: Ted Baker
Sent: Friday, January 19, 2001 6:33 AM

| ... it is very likely that the two records would end up interspersed, and
| they could not be read by any means at all ...

You seem to have missed what is atomic here.  It is not the
writing of a block of data as an atomic block.  What is atomic is
the positioning to the end being atomic with the writing of
whatever is the (implementation-dependent) atomically writable
unit of data.

What you gain is that you will NEVER OVERWRITE anything, because
the positioning is atomic with the writing.  Thus, no data will be
lost, even if some other thread of control tries to change the
file position.

| ... atomicity of O_APPEND is not useful for Stream_IO
| files ...

OK. You are correct that there may be interleaving, and that
argues well against the utility of this feature with Ada streams.
The places where the append semantics are most useful are where
data is being written in chunks (a buffer full), and the
maximum atomically writeable chunk will be implementation defined
(usually at least 512 bytes on disk files).

****************************************************************

From: Robert Dewar
Sent: Friday, January 19, 2001 6:45 AM

<<But on the other hand, it is perfectly possible to write to a Stream_IO file
using only Stream_IO.Write (perhaps because you do conversion to byte arrays
yourself), in which case the atomicity property of O_APPEND is applicable.
>>

Right, same point I just made (sorry for duplication), but you have got to
admit that in practice it is pretty unlikely that anyone is actually going
to this approach.

****************************************************************

From: Ted Baker
Sent: Saturday, January 20, 2001 10:30 AM

| Robert Dewar wrote:
| > ... I guess at this stage, if we really want to
| > allow the O_APPEND implementations to be considered conforming, we just
| > bite the bullet and say that it is implementation defined whether
| > positioning is allowed for stream files in append mode -- seems pretty
| > ugly, but I see no alternative.

| We should be careful here to allow what to me is the most natural
| expectation, namely that positioning is allowed, but the position is
| relative to the part you are adding.

That is really strange!  What is "the part you are appending"????
Logically, each write operation to the file appends a new "part".

Since the standard is effectively silent on the definition of
"append", but uses the term to say that in Append_File mode "only
appending" is to be peformed, it is very clear that the dictionary
definition must apply.  That means each write operation to the file
must add on the output to the end of the file: no overwriting or
truncation is possible.

To accomplish this you have two choices:

1) Allow the reader to do "positioning", as POSIX does, but
require that this repositioning have no effect on where the output
goes; the data still gets written at the end of file.

2) Forbid repositioning on files in Append_File mode.

Both of these allow (but to not require) the implementation to
use O_APPEND mode.

****************************************************************

From: dewar@GNAT.COM
Sent: Saturday, January 20, 2001 10:41 AM

<<Since the standard is effectively silent on the definition of
"append", but uses the term to say that in Append_File mode "only
appending" is to be peformed, it is very clear that the dictionary
definition must apply.  That means each write operation to the file
must add on the output to the end of the file: no overwriting or
truncation is possible.>>

This is quite misleading, Ted, have you read the whole thread here.
TO argue that the RM is clear and unambiguous at this point is very
dubious.

In particular, Append_File mode IS defined at several places to mean
that the effect is to position to the end of the file, and there appears
to be a quite deliberate exclusion of stream_io (the only case nder
discussion -- please remember this) from the statement that all writes
are to the end of the file.

Please read again my description of why my model of appending is perfectly
reasonable according to the dictionary definition, (it's the analogy
of writing things by hand in response to a suggestion that you append
a note to the end of what's written -- there is no requirement that
your appended note be written in a compeltely sequential manner.

Furthermore, we have this VERY important point that in the normal
mode of operation of Stream_IO, namely using the stream attributes,
the whole idea of process-safe appending is broken in any case.

****************************************************************

From: Ted Baker
Sent: Saturday, January 20, 2001 11:41 AM

| This is quite misleading, Ted, have you read the whole thread here.

I've gone back and I think I've read all of it, though I have read it
pretty much in reverse order, due an intervening busy period here.

| In particular, Append_File mode IS defined at several places to mean
| that the effect is to position to the end of the file, and there appears

I used emacs to search through the RM for such places, before my last
e-mail.  Not much is said:

Append_File:

| ... These values correspond respectively to the cases where only
| reading, only writing, or only appending are to be performed.
                            ==============

This seems to be the most definitive statement of the semantics
of Append_File.

Open:

| ... When the file is opened with mode Append_File, transfer to the file
| starts after the last element of the file.

This simply confirms what append means.

Close:

| ... for sequential files, if the file being closed has mode Out_File or
| Append_File, then the last element written since the most recent open
| or reset is the last element that can be read from the file.

This confirms that if one somehow were able to overwrite
a portion of the file in Append_File mode, and that were the
last operation done, the rest of the file must be truncated.
To me, this further underscores the requirement that in that mode
all output must be to the end of the file, i.e., either there
is no repositioning, or positioning has no effect.

Reset:

| ... writing to its elements can be restarted ...  after the last
| element of the file (for mode Append_File).  ...  In addition, for
| sequential files, if the given file has mode Out_File or
| Append_File when Reset is called, the last element written since
| the most recent open or reset is the last element that can be read
| from the file.

Se the comment above on Close.

Write:

|         6   Operates on a file of mode Out_File or Append_File.  Writes the
|         value of Item to the given file.

Adds nothing special for Append_File.

| 17   (20) Append_File mode is not supported for the generic package
|      Direct_IO.

Further supports the idea that Append_File cannot be used in
conjunction with random-access/respositioning.

[skipping stuff that relates only to text I/O, page and line lengths, etc.]

| Set_Mode:

| ...  If the new mode is Append_File, the file is positioned to its
| end ...

Consistent with the requirement that all operations can only
append to the file.

| to be a quite deliberate exclusion of stream_io (the only case nder
| discussion -- please remember this) from the statement that all writes
| are to the end of the file.

Where do you see this as an exclusion?

| 10  type File_Mode is (In_File, Out_File, Append_File);
|     --  for Sequential_IO, Text_IO, Wide_Text_IO, and Stream_IO

|         11   These values correspond respectively to the cases where only
|         reading, only writing, or only appending are to be performed.
                                    ==============
|         12   The mode of a file can be changed.

| Please read again my description of why my model of appending is perfectly
| reasonable according to the dictionary definition, (it's the analogy
| of writing things by hand in response to a suggestion that you append
| a note to the end of what's written -- there is no requirement that
| your appended note be written in a compeltely sequential manner.

That is a big stretch, especially given the explicit statements
about the truncation that happens when you close or reset the file
(see above) if the last element written is not at the end of the
file.  That requires at least some restriction on order of operations.

| Furthermore, we have this VERY important point that in the normal
| mode of operation of Stream_IO, namely using the stream attributes,
| the whole idea of process-safe appending is broken in any case.

Granted you can't get process-safe appending in that case, but I
believe you still ought to allow (not requir) an implementation to
use O_APPEND to implement this mode.  The way to do that is to
either completely disallow positioning for files in Append_File mode,
or to specify that such positioning has no effect on output of
such files-- the output still is appended (to the end).

****************************************************************

From: dewar@GNAT.COM
Sent: Saturday, January 20, 2001 11:53 AM

<<Open:

| ... When the file is opened with mode Append_File, transfer to the file
| starts after the last element of the file.
>>

No, in RM terms, it *defines* what Append mode means. Since this is the
only defined effect, this is the ONLY effect. That's the way the RM
works.

<<| ... for sequential files, if the file being closed has mode Out_File or
| Append_File, then the last element written since the most recent open
| or reset is the last element that can be read from the file.

This confirms that if one somehow were able to overwrite
a portion of the file in Append_File mode, and that were the
last operation done, the rest of the file must be truncated.
>>

This really rates a huh? :-)

This is a statement about *SEQUENTIAL FILES* we are not talking about
sequential files! We are talking about stream files. Indeed it is the
specific exclusion of stream files that is significant here. If the
authors of the RM had intended your interpretation, they would have
omitted the qualifer "for sequential files".

Thus this last quote, which you seem to think supports your position,
in fact is the strongest evidence internal to the RM that supports the
position you disagree with!

Ted, please focus on the fact that this entire discussion is about stream
files. That's an important point here, because we all agree that sequential
files COULD be handled with O_APPEND, though of course there is no such
requirement, and any program that depends on this is relying on undocumented
implementation dependent (not even implementation defined) behavior.

****************************************************************

From: dewar@GNAT.COM
Sent: Saturday, January 20, 2001 10:57 AM

<<1) Allow the reader to do "positioning", as POSIX does, but
require that this repositioning have no effect on where the output
goes; the data still gets written at the end of file.>>

There is no conceivable justification for this interpretation from the
definitions in the RM.

****************************************************************

From: Ada Rapporteur Group of WG9; managing the Ada issues [ARG@ACM.ORG]
on behalf of dewar@GNAT.COM
Sent: Saturday, January 20, 2001 12:45 PM
To: ARG@ACM.ORG
Subject: Re: AI-85 analysis and discussion.

<<11   These values correspond respectively to the cases where only
reading, only writing, or only appending are to be performed.
>>

Yes, but that says nothing, it does not define what appending means,
only that this is what happens after such an open. So we have to lool
elsewhere in the RM for what it means.

And we find that for stream files, it means that an initial open positions
you at the end, as does a reset, and for sequential files, that writes
go at the end.

But there is NO statement that writes for stream files go at the end.
Your only argument for this is that you think it should be this way,
but there is nothing in the RM to support this view. Now we can
speculate, as Pascal has interestingly done, on historical reasons
why the RM is the way it is, and perhaps that it does not map the
original intent, which is always a fair discussin.

But your argument that the RM clearly requires stream file writes to
be done at the end definitely does NOT hold water.

<<   28   The subprograms Create, Open, Close, Delete, Reset, Mode, Name, Form,
                                ===========
   Is_Open, and End_of_File have the same effect as the corresponding
                                 ===============
   subprograms in Sequential_IO (see A.8.2).
>>

Yes, indeed, but unfortuantely we are talking about the effect of write,
and positioning operations, which are NOT included in the above list.

Yes, Open *DOES* have the same effect.

As for close, the definition of close is:

        10   Severs the association between the given file and its associated
        external file.  The given file is left closed.  In addition, for
        sequential files, if the file being closed has mode Out_File or
        Append_File, then the last element written since the most recent open
        or reset is the last element that can be read from the file.  If no
        elements have been written and the file mode is Out_File, then the
        closed file is empty.  If no elements have been written and the file
        mode is Append_File, then the closed file is unchanged.

And we see very clearly that the rule about last element written being
the last element applies only for sequential files. It could not be
much clearer!

Yes, it is a bit odd to say that the behavior is the same for A and B,
when the rule you are citing specifically says that A and B behave
differently, and in Pascal's mode, you could site this as additional
evidence that the phrase "for sequential files" here was from the
time when stream files were considered to be sequential files, and
someone forgot to fix it.

Also, how can you explain, if your meaning is so self-evident, that there
is no provision for preventing positioning operations in append mode, if
they are completely meaningless, as you contend.

I am not saying that your desired interpretation is unreasonable, it is
indeed one of the positions we are considering, but to argue that the
RM unambguously specifies this behavior (and thus presumably requires
no modification in this area) is just not supported by the quoted text.

It is also not clear to what extent you are arguing with O_APPEND in
mind. Obviously the RM does not require O_APPEND behavior in any case,
and indeed, as we have discussed, O_APPEND and stream_io are not very
compatible, in that the normal mode for stream io is lots o little
writes which would break the guarantee of indivisibility in any case.

Given that O_APPEND is not *required*, the question is whether to *allow*
it at all. There are several possible answers:

1. No O_APPEND cannot be used, because positioning must be possible.

2. (more reasonable). Of coruse O_APPEND can be used, but if someone
does a positioning operation, then you have to implicitly reopen in
update mode. By the way, in GNAT we do this kind of reopening anyway
on a reset, because you don't want to have to open in update mode
just in case you do writes.

3. No positioing is allowed, so O_APPEND is OK, this has two subcases

  3a. We make it clear that a given implementation is allowed to
      restrict positioning on any files it cares to. Two subcases

      3a1. On all files (Tuck's view, I disagree)
      3a2. On stream files in append mode

  3b. We rewrite the RM to prohibit positioning on stream files in
      append mode (or if you take Ted's view, no rewriting is needed
      because he thinks that's what the RM says now).

4. (Least convincing, though perhaps this is what Ted thinks the RM
   says now). Positioning on stream files is allowed but does not
   affect subsequent write operations.

We need to

a) choose one of these positions

b) figure out what the RM should be saying to support this position

I really don't think it is very helpful to argue that both the ltter
and intent of the RM clearly specify *ANY* of these positions, because
if they did, we would not be having this discussion.

****************************************************************

From: Ted Baker
Sent: Saturday, January 20, 2001 12:23 PM

I see you chose to reply only to the parts of my message
where you could pick fault.  I guess that means you agreed
with the rest.

| <<Open:

| | ... When the file is opened with mode Append_File, transfer to the file
| | starts after the last element of the file.
| >>

| No, in RM terms, it *defines* what Append mode means. Since this is the
| only defined effect, this is the ONLY effect. That's the way the RM
| works.

No.  It only defines part of what Append mode means.  We also have:

    --  for Sequential_IO, Text_IO, Wide_Text_IO, and Stream_IO
                                                      =========
        11   These values correspond respectively to the cases where only
        reading, only writing, or only appending are to be performed.
                                  ==============

| <<| ... for sequential files, if the file being closed has mode Out_File or
| | Append_File, then the last element written since the most recent open
| | or reset is the last element that can be read from the file.

| This confirms that if one somehow were able to overwrite
| a portion of the file in Append_File mode, and that were the
| last operation done, the rest of the file must be truncated.
| >>

| This really rates a huh? :-)
| This is a statement about *SEQUENTIAL FILES* ...

Huh? :-)  Read your RM again, the section on Stream_IO:

   28   The subprograms Create, Open, Close, Delete, Reset, Mode, Name, Form,
                                ===========
   Is_Open, and End_of_File have the same effect as the corresponding
                                 ===============
   subprograms in Sequential_IO (see A.8.2).
                  =============

| Ted, please focus on the fact that this entire discussion is about stream
| files. That's an important point here, because we all agree that sequential
| files COULD be handled with O_APPEND ...

Of course.  However, I think we all agree that the RM did not do a
good job of explicitly stating the semantics of Append_File with
stream files.  Therefore, I believe it is appropriate to appeal to
analogies with other uses of Append_File.

The RM text on stream files itself refers back to the concepts of
sequential and direct files.  In addition to the cited text above,
which refers back to Sequential_IO for the effects of Open and Close,
we have: "Access to a stream file is either
sequential, via a call on Read or Write to transfer an array of
stream elements, or positional (if supported by the implementation
for the given file), by specifying a relative index for an
element."

It seems clear to me that mode Append_File only makes sense when
the access is sequential, since it is not allowed for direct
files.

****************************************************************

From: dewar@GNAT.COM
Sent: Saturday, January 20, 2001 12:47 PM

<<I see you chose to reply only to the parts of my message
where you could pick fault.  I guess that means you agreed
with the rest.>>

you guess wrong! I was responding only to the specfic quotes from the
RM, because that's the only relevant issue at this stage if we are
talking about what the RM means. The fact that Ted Baker thinks it
should mean something else or that Robert or Pascal or anyone else
does, is not really relevant to the issue of trying to clean this up.

****************************************************************

From: Ted Baker
Sent: Saturday, January 20, 2001 1:49 PM

| Yes, it is a bit odd to say that the behavior is the same for A and B,
| when the rule you are citing specifically says that A and B behave
| differently, and in Pascal's mode, you could site this as additional
| evidence that the phrase "for sequential files" here was from the
| time when stream files were considered to be sequential files, and
| someone forgot to fix it.

Yes, I suppose Pascal and I are arguing in the same direction
here, i.e., that spelling out the semantics of Append_File mode
for stream files was unintentionally neglected when stream files
were extended to allow non-sequential access.  What we have now
for stream files is a union of sequential and direct file operations.
My contention is that where there is a gap in the description
of stream files the semantics for sequential files should
be applied to the sequential io operations, and the semantics for
direct files should be applied to the direct io opeations.

| I am not saying that your desired interpretation is unreasonable, it is
| indeed one of the positions we are considering, but to argue that the
| RM unambguously specifies this behavior (and thus presumably requires
| no modification in this area) is just not supported by the quoted text.

I'm willing to concede that I overstated the support in the ARM
for this view, and that some clarification is needed.

| Given that O_APPEND is not *required*, the question is whether to *allow*
| it at all. There are several possible answers:

| 1. No O_APPEND cannot be used, because positioning must be possible.

I've been arguing against the above.

| 2. (more reasonable). Of coruse O_APPEND can be used, but if someone
| does a positioning operation, then you have to implicitly reopen in
| update mode. By the way, in GNAT we do this kind of reopening anyway
| on a reset, because you don't want to have to open in update mode
| just in case you do writes.

I would not object to this strongly, but it seems to violate the
rule of least surprise, i.e., by making repositioning act like a reopen.

| 3. No positioing is allowed, so O_APPEND is OK, this has two subcases

|   3a. We make it clear that a given implementation is allowed to
|       restrict positioning on any files it cares to. Two subcases

|       3a1. On all files (Tuck's view, I disagree)

I guess you don't disagree that we can't reposition a UNIX
terminal device (character special file).

|       3a2. On stream files in append mode

Seems reasonable to me.

|   3b. We rewrite the RM to prohibit positioning on stream files in
|       append mode (or if you take Ted's view, no rewriting is needed
|       because he thinks that's what the RM says now).

This is what I would prefer, because I think that is most consistent
with the rest of what is in the RM.

| 4. (Least convincing, though perhaps this is what Ted thinks the RM
|    says now). Positioning on stream files is allowed but does not
|    affect subsequent write operations.

I think this would also be OK, but is less consistent with what
the RM says.

| We need to

| a) choose one of these positions

| b) figure out what the RM should be saying to support this position

| I really don't think it is very helpful to argue that both the ltter
| and intent of the RM clearly specify *ANY* of these positions, because
| if they did, we would not be having this discussion.

OK.  It is not clear enough, and needs to be specified.

****************************************************************

From: dewar@GNAT.COM
Sent: Saturday, January 20, 2001 2:12 PM

<<I guess you don't disagree that we can't reposition a UNIX
terminal device (character special file).>>

No, I agree with that (and indeed interpreted the phrase about files
not allowing positioning to be *exclusively* talking about this case :-)

<<I would not object to this strongly, but it seems to violate the
rule of least surprise, i.e., by making repositioning act like a reopen.>>

The argument in favor is that it is upwards compatible for implementations
that do allow positioning (and I think we have to agree that the current
RM wording does *allow* positioning oeprations on stream files in append
mode since it does not disallow it).

A nasty mess.

I must say I really VERY much doubt that anyone is using the native
write operations in append mode on an O_APPEND implementation today
to ensure process serialization of output. It is just *so* convenient
to use the stream attributes, and that's what one does 99.9% of the time.
It is a real pain to do otherwise (you have to instantiate your own
implementation of streams that generates buffers that can then be
setup using stream attributes, and then manually write these buffers
with direct calls to Stream_IO.Write -- yes it can be done, but I would
be surprised if it is done.

Indeed it seems rather error prone to me to even suggest that O_APPEND
mode will be used for append mode with streams, since it is SO easily
misused.

Interestingly, in GNAT we don't use O_APPEND for anything (though we do
have a general mechanism for doing Ada I/O operations on files that are
attached to streams opened with C calls, so someone can do this for them
selves). We don't even use O_APPEND for sequential files (there is no
requirement to do so), and we never got a suggestion to change this, let
alone a problem report.

****************************************************************


Questions? Ask the ACAA Technical Agent