!standard A.8.2 (16) 05-01-25 AI95-00085/09 !standard A.12.1(28.1) !standard A.12.1(33) !class confirmation 95-08-19 !status Amendment 200Y 02-07-09 !status WG9 Approved 02-06-21 !status ARG Approved 6-0-1 02-02-12 !status work item 98-10-09 !status ARG Approved (subject to editorial review) 12-0-0 98-04-03 !status work item 95-11-01 !status received 95-08-19 !priority High !difficulty Hard !subject Append_File, Reset, and positioning for Stream_IO !summary A call to Reset without the Mode parameter is equivalent to a call to Reset with the current mode of the file. The beginning and end of the file mentioned in the description of the semantics of Reset designate the beginning and end of the external file. It is legitimate for an implementation to raise Use_Error on the positioning operations of Stream_IO if the underlying operating system does not support positioning. The procedure Stream_IO.Set_Mode sets the mode of the file, it doesn't necessarily change it. !question 1. Are the following two calls equivalent, even if F is opened in Append_File mode? (Yes.) Reset (F, Mode => Mode (F)); Reset (F); 2. A call to Reset with mode In_File allows reading to be restarted "from the beginning of the file". A.7(2) indicates that the term "file" refers to a file object, not to an external file. Does the phrase "beginning of the file" refer to the beginning of the external file? (Yes.) 3. For a Unix implementation that uses the Unix append mode to implement files opened in Append_File mode, what are the possible effects of Set_Index? (Do the right thing or raise Use_Error.) 4. A.12.1(28.1/1) says that "the Set_Mode procedure changes the mode of file." Does that mean that Set_Mode can only be used to "change" the mode of the file? (No.) !response 1. A.8.2(16) states that: "if a Mode parameter is supplied, the current mode of the given file is set to the given mode." This paragraph doesn't mention any other effect for the Mode parameter; therefore, setting the Mode parameter Mode (F) has essentially no effect, and the two calls Reset (File, Mode (File)) and Reset (File) are equivalent. For a file which is opened in Append_File mode, Reset (F) positions the file so that writing can be restarted after the last element of the external file. This is explicitly stated in A.8.2(16), although this section uses the word "file" when it should say "external file". Note that for a Unix implementation, it makes it possible to use the Unix append mode, and to implement any of these two calls without closing and reopening the file. 2. When A.8.2(16) talks about the "beginning of the file" or "the last element of the file" it is talking about the beginning and end of the external file (despite A.7(2)). For a file that is Reset to mode In_File, reading restarts at the beginning of the external file. This is supported by the annotation in AARM A.10.2(4.a), which indicates that Reset should be similar to closing and then reopening. The internal file essentially disappears between closing and reopening, so clearly Reset is resetting based on properties of the external file, not of the internal file. Note that for a Unix implementation which uses the Unix append mode to implement files opened in Append_File mode, this means that resetting to In_File or Out_File must be implemented by closing and reopening. 3. It is the intent that a Unix implementation can use the Unix append mode to implement files opened in Append_File mode. For a stream designating such a file, positioning is effectively impossible because, if the O_APPEND bit was set at open time, "the file pointer is set to the end of the file prior to each write" (excerpt from the Unix man page for open). A.12.1(33) makes it clear that such an implementation is legitimate, provided that it raises Use_Error on the positioning operations: "if positioning is not supported for the given file, then a call to Index or Set_Index propagates Use_Error." The standard makes no requirements about whether a particular file supports positioning or not. Thus, an implementation does not have to support positioning on any files at all if it likes. It may use any rule that is appropriate for the target system. In particular, it is acceptable for a file to not support positioning if the mode is Append_File, and for the same external file to support positioning if the mode is Out_File. Although this compromises portability (different compilers may use different implementations, and certainly different OSes may have different restrictions), the stress here is on performance: the intent is that the file and stream operations should be implemented as a thin layer on top of the OS calls, and should not have to simulate services that are not provided by the underlying OS. Moreover, since there is no requirement to support positioning for any stream files, a truly portable program would have to avoid positioning a stream file in any mode. Thus, allowing implementations to use O_APPEND does not impose any further portability penalty. 4. A.12.1(28.1/1) should say that "the procedure Set_Mode sets the mode of the file." If the Mode parameter is the current mode of the file, then surely the mode of the file is not changed. !corrigendum A.8.2(16) @drepl Resets the given file so that reading from its elements can be restarted from the beginning of the file (for modes In_File and Inout_File), and so that writing to its elements can be restarted at the beginning of the file (for modes Out_File and Inout_File) or after the last element of the file (for mode Append_File). In particular, for direct access this means that the current index is set to one. If a Mode parameter is supplied, the current mode of the given file is set to the given mode. In addition, for sequential files, if the given file has mode Out_File or Append_File when Reset is called, the last element written since the most recent open or reset is the last element that can be read from the file. If no elements have been written and the file mode is Out_File, the reset file is empty. If no elements have been written and the file mode is Append_File, then the reset file is unchanged. @dby Resets the given file so that reading from its elements can be restarted from the beginning of the external file (for modes In_File and Inout_File), and so that writing to its elements can be restarted at the beginning of the external file (for modes Out_File and Inout_File) or after the last element of the external file (for mode Append_File). In particular, for direct access this means that the current index is set to one. If a Mode parameter is supplied, the current mode of the given file is set to the given mode. In addition, for sequential files, if the given file has mode Out_File or Append_File when Reset is called, the last element written since the most recent open or reset is the last element that can be read from the external file. If no elements have been written and the file mode is Out_File, the reset file is empty. If no elements have been written and the file mode is Append_File, then the reset file is unchanged. !corrigendum A.12.1(28.1/1) @drepl The Set_Mode procedure changes the mode of the file. If the new mode is Append_File, the file is positioned to its end; otherwise, the position in the file is unchanged. @dby The Set_Mode procedure sets the mode of the file. If the new mode is Append_File, the file is positioned to its end; otherwise, the position in the file is unchanged. !appendix !section A.7(02) !subject Questions about Append_File mode !reference RM95-A.7(2) !reference RM95-A.8.2(15,16) !reference AARM95-A.10.2(4.a) !from Keith Thompson 95-08-09 !reference as: 95-5253.a Keith Thompson 95-8-9>> !discussion Some aspects of Append_File mode seem unclear. For example: 1. Reset with an explicit Mode => Append_File (for a text or sequential file) causes writing to continue at the end of the file, yes? Assume F is a (text or sequential) file whose current mode is Append_File. Are the following two calls equivalent? Reset(F, Mode => Append_File); Reset(F); More generally, are these always equivalent? Reset(F, Mode => Mode(F)); Reset(F); I believe they should be, but I've been unable to prove it from the RM or AARM. 2. A call to Reset with mode In_File allows reading to be restarted "from the beginning of the file". RM95-A.7(2) indicates that the term "file" refers to a file object, not to an external file. Does the phrase "beginning of the file" refer to the beginning of the external file? In particular, what is the effect of the following code fragment, assuming all operations are successful? declare package Int_IO is new Ada.Sequential_IO(Integer); F: Int_IO.File_Type; File_Name: constant String := "foo.dat"; begin Int_IO.Create(F, File_Name); for I in 1 .. 3 loop Int_IO.Write(F, I); end loop; Int_IO.Close(F); -- -- foo.dat now contains (1, 2, 3) -- Int_IO.Open(F, Mode => Int_IO.Append_File, Name => File_Name); for I in 4 .. 6 loop Int_IO.Write(F, I); end loop; -- -- foo.dat now contains (1, 2, 3, 4, 5, 6), -- but where is the "beginning of the file"? -- Int_IO.Reset(F, Mode => Int_IO.Out_File); Int_IO.Write(F, 999); Int_IO.Close(F); end; If the "beginning of the file" is the beginning of the external file, the final contents of foo.dat should be the single integer 999. If it refers to the beginning of the file object, *and* the beginning of a file in Append_File mode is considered to be the point at which it was opened, foo.dat should contain 4 integers: 1, 2, 3, and 999. (The previously written values 4, 5, and 6 are lost in either case). It would be easier and more intuitive, IMHO, for Reset to go to the beginning of the external file, but I can't infer this behavior from the RM. The note in AARM95-A.10.2(4.a): The behavior of Reset should be similar to closing a file and reopening it with the given mode. tends to support this, but it's not binding. On the other hand, if Reset goes back to the beginning of the external file, there's no easy way to go back to the position at which it was opened for appending. (This is probably no great loss.) **************************************************************** !section A.8.2(16) !subject Questions about Append_File mode !reference AI95-00085 !reference RM95-A.8.2(16) !reference RM95-A.7(2) !reference AARM-A.10.2(4.a) !from Tucker Taft 95-10-30 !reference 95-5373.h Tucker Taft 95-10-30>> !discussion I certainly believe that Reset(File, Mode(File)) is equivalent to simply Reset(File). I can't imagine any other meaningful interpretation for A.8.2(16). As far as external vs. internal file, I believe that the wording in A.8.2(16) is sloppy, and whenever it talks about the "beginning of the file" or "the last element of the file" it is talking about the beginning and end of the external file (despite A.7(2)). It makes no difference whether the file was initially opened in append mode, or was reset to that mode somewhere along the way. The annotation in A.10.2(4.a) seems to make this intent clear, by indicating that Reset should be similar to closing and then reopening. The internal file essentially disappears between closing and reopening, so clearly Reset is resetting based on properties of the external file, not of the "internal" file. **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 8:36 AM Subject: Re: AI95-00085 <> I find this FAR too vague. Obviously if poisitioning is not supported at all (a system with only tapes :-) then no argument. But how about a unix-like system in which general positioning is of course supported, but you *prefer* to use some specific OS facility to implement e.g. append mode files, which does not support the positioning that is needed. Are you allowed to blow away the call with a use_error. I think the answer should be no! There should be maximum pressure to make your IO system conform to the Ada model, and I dislike strong suggestions to the contrary. I think this paragraph should be removed from the AI. It adds nothing, except an unwelcome suggestion that this kind of compromise is perfectly OK. **************************************************************** From: Pascal Leroy[SMTP:phl@Rational.Com] Sent: Thursday, April 09, 1998 10:55 AM Subject: Re: AI95-00085 > But how about a unix-like system in which general positioning is of > course supported, but you *prefer* to use some specific OS facility > to implement e.g. append mode files, which does not support the > positioning that is needed. Are you allowed to blow away the call > with a use_error. I think the answer should be no! But the feeling at the last meeting was precisely that the implementation should be allowed to use Unix O_APPEND, and that if it does so it probably has to raise Use_Error on a call to Set_Index for a stream that is open in Append_Mode. It seems to me that requiring an implementation to support Set_Index on a stream in Append_Mode effectively precludes use of O_APPEND, and that's annoying. Pascal **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 9:10 AM Subject: Re: AI95-00085 <> Why is that annoying? If people want to use Unix-specific I/O, please do it directly! The whole idea in defining an I/O facility at the language level is for portability. Yes, it is frustrating when the models do not match, but that's too bad, since models between operating systems do not match precisely, this is inevitable. I think the decision to allow O_APPEND here is plain wrong. It is perfectly possible to implenment the Ada I/O facilities without allowing O_APPEND. **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 9:15 AM Subject: Re: AI95-00085 Incidentally, I absolutely agree that the design of the Ada with respect to Append was injudicious. I suspect that whoever did the design simply did not take the effort to look at the Unix semantics. But there are only two acceptable approaches in my mind at this stage: 1. Change the language to match the Unix semantics 2. Insist on the (perfectly possible to meet even on Unix) requirement that the implementation implement the language as chosen. The permission not to implement things in the RM is supposed to handle cases where it is impractical or impossible to implement things right, not simply cases where implementing things right is, to use Pascal's word, "annoying". **************************************************************** From: Pascal Leroy[SMTP:phl@Rational.Com] Sent: Thursday, April 09, 1998 11:43 AM Subject: Re: AI95-00085 > 1. Change the language to match the Unix semantics Well, if you ask me, the notion of calling Set_Index on a stream in Append_Mode if pretty weird, and it's not clear what the semantics of such a call should be. Say that after calling Set_Index, I call Write: what happens? Does it write at the end of the file? At the current point? Does it truncate the file? At the very least, these questions require clarification, and this is not changing the language. I am tempted to say that Set_Index in Append_Mode is likely to be a programming error, and should always raise some exception (and I now regret that I didn't write the AI in this direction in the first place). > 2. Insist on the (perfectly possible to meet even on Unix) requirement > that the implementation implement the language as chosen. > > The permission not to implement things in the RM is supposed to handle > cases where it is impractical or impossible to implement things right, > not simply cases where implementing things right is, to use Pascal's > word, "annoying". I was not thinking in terms of implementation, but in terms of usage. We have many users who want to mix Ada and C I/Os on the same file. I would find it rather unpleasant if an Ada Append_Mode file was not opened with O_APPEND, or if passing an O_APPEND file to Ada caused it to be closed and reopened so that Ada can do its silly simulation. Pascal **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 9:46 AM Subject: Re: AI95-00085 <> Mixing Ada and C I/O may or may not succeed. It is not a requirement that it work smoothly, just something nice to have. You are free to implement a form parameter to ensure this smooth interchange. As I am sure you know GNAT takes a different tack, we allow a file to be opened using the stream identifier passed in from C, and that ensures a common view of the file. **************************************************************** From: Robert I. Eachus[SMTP:eachus@mitre.org] Sent: Thursday, April 09, 1998 10:10 AM Subject: Re: AI95-00085 At 10:15 AM 4/9/98 EDT, Robert Dewar wrote: >The permission not to implement things in the RM is supposed to handle >cases where it is impractical or impossible to implement things right, >not simply cases where implementing things right is, to use Pascal's >word, "annoying". Is an implementation allowed to silently close and reopen an (external) file in response to a Set_Position, if it was originally opened in Append mode? I assume so, and that this would be the preferable option for Unix. (I'm not sure what the right thing to do is if an unnamed file/pipe was opened in append mode though. Probably, that is a case for Use_Error.) Robert I. Eachus with Standard_Disclaimer; use Standard_Disclaimer; function Message (Text: in Clever_Ideas) return Better_Ideas is... **************************************************************** From: Tucker Taft Sent: Thursday, April 09, 1998 10:42 AM Subject: Re: AI95-00085 Another possibility would be to use O_APPEND mode until a Set_Index is called, and only then go through the trouble of re-opening in non-append mode rather than raising Use_Error, with the presumption that Set_Index will be used rarely for files opened in append mode. Effectively Set_Index would be almost like a Reset(F, Mode => Out_File) (though of course Mode(F) would still return Append_File). -Tuck **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 11:50 AM Subject: Re: AI95-00085 < Out_File) (though of course Mode(F) would still return Append_File). >> That's perfectly reasonable. We do something like this in GNAT when you switch from read to write mode for stream_io, we first open in read only mode, then reopen in update mode when you switch. **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 11:47 AM Subject: Re: AI95-00085 << Is an implementation allowed to silently close and reopen an (external) file in response to a Set_Position, if it was originally opened in Append mode? I assume so, and that this would be the preferable option for Unix. (I'm not sure what the right thing to do is if an unnamed file/pipe was opened in append mode though. Probably, that is a case for Use_Error.) >> Of course you are allowed to silently close and reopen in response to Set_Position (you can do it on every read if you like :-) **************************************************************** From: Randy Brukardt[SMTP:Randy@rrsoftware.com] Sent: Thursday, April 09, 1998 12:29 PM Subject: RE: AI95-00085 I've been following this discussion, and I just don't get it. Probably the problem is my lack of experience with the common usages of Unix. I just don't understand why anyone would want to use O_APPEND on a positionable device. When I studied the Unix documents, I just viewed O_APPEND as (to borrow Robert's term) a junk mode. Why would anyone WANT to disallow positioning on their file? I can see that it may not matter in some cases, but I can't see any reason that anyone would want the defined behavior of O_APPEND. I would expect that most users just want to start writing at the end of the file, and don't care about positioning. I would expect that non-positionable devices would essentially be in append mode at all times, so there is no need for some junk mode in that case, either. The Ada Append_Mode matches the behavior I would expect exactly: starting writing at the end of the file, but has no other semantic effects. If this means not using a junk mode, so be it. It was always my view that Append is a feature of Open/Create, not a mode. It should have been implemented as an optional parameter to Open/Create. As far as mixing Ada and C I/O, I don't see a problem. If the file is opened in C, and imported into Ada somehow, there cannot be a problem. That is outside of anything in the Ada standard, so do whatever you want - raise Use_Error anywhere you have to. If the file is opened in Ada and passed to C, there shouldn't a be a problem. Since O_APPEND disallows various operations, I don't see how not using it on a file could break any existing C code (unless that C code was completely broken in the first place). In any, I would expect most code that takes files to work irregardless of the file mode. If you absolutely have to have O_APPEND opens, add a form parameter "Open_with_junk_O_Append" and open it that way. Then you can raise Use_Error anywhere you like. I agree with Robert that there is no good reason to break portability of software by allowing compilers to raise Use_Error when it is easy to avoid it. This is especially so in order to support junk file options of a particular (admittedly popular) operating system. Randy. **************************************************************** From: Tucker Taft[SMTP:stt@inmet.com] Sent: Thursday, April 09, 1998 2:14 PM Subject: RE: AI95-00085 > I've been following this discussion, and I just don't get it. Probably the > problem is my lack of experience with the common usages of Unix. > > I just don't understand why anyone would want to use O_APPEND on a > positionable device. When I studied the Unix documents, I just viewed > O_APPEND as (to borrow Robert's term) a > junk mode. Why would anyone WANT to disallow positioning on their file? I > can see that it may not matter in some cases, but I can't see any reason > that anyone would want the defined behavior of O_APPEND. I would expect > that most users just want to start writing at the end of the file, and > don't care about positioning. O_APPEND is a relatively new mode for Unix (speaking geologically here). It was invented to solve the problem of having multiple processes all writing to the same log file at the same time, wanting all messages to end up at the end, no matter which process wrote them. Without O_APPEND, each process maintains its own internal idea of what is its position in the file, and even though it may position to the end when it first opens the file, because other processes are writing to the same file simultaneously, its position may no longer correspond to the end when it gets around to writing. In my view, the "Append_File" mode in Ada is intended to allow the use of O_APPEND, but not require it, at least for Sequential_IO. However, A.14 makes it clear that each stream file object has its own current_index, presuming the file supports positioning. This would seem to preclude the use of O_APPEND, with its automatic position-to-the-end semantics, unless positioning is disallowed for the file. Hence it seems necessary to either raise Use_Error on Set_Index, or not use O_APPEND mode for stream files. Presumably a Form parameter could be used to control which semantics are preferred. > I would expect that non-positionable devices would essentially be in append > mode at all times, so there is no need for some junk mode in that case, > either. > > The Ada Append_Mode matches the behavior I would expect exactly: starting > writing at the end of the file, but has no other semantic effects. If this > means not using a junk mode, so be it. > > It was always my view that Append is a feature of Open/Create, not a mode. > It should have been implemented as an optional parameter to Open/Create. The issue is what happens in the presence of multiple processes all appending to the same file. > I agree with Robert that there is no good reason to break portability of > software by allowing compilers to raise Use_Error when it is easy to avoid > it. This is especially so in order to support junk file options of a > particular (admittedly popular) operating system. It seems better to support either approach by use of a Form parameter. It is not clear to me which should be the default for Append_File in the absence of a Form parameter, but it now seems clear to me that O_APPEND mode is inconsistent with the A.14(4) requirement that each stream file maintains its own current index, unless positioning is not allowed for the file (i.e. Set_Index raises Use_Error). > Randy. -Tuck **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Thursday, April 09, 1998 2:41 PM Subject: Re: AI95-00085 This kind of Ada-centric view (of append-only file access modes) is typical of the kind of attitude that has held Ada back. The Ada'95 append mode was intended to allow access to OS-provided append modes. If we now prevent it from mapping in a sensible way, we just make Ada look absurd. --Ted **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 2:40 PM Subject: Re: AI95-00085 <> Are you specifically talking about O_APPEND here? We may say that the Ada 95 append mode was intended to allow access to OS-provided append modes, but all OS's do not provide the same stuff, are you saying that you think append mode should be entirely implementation dependent? **************************************************************** From: Randy Brukardt[SMTP:Randy@rrsoftware.com] Sent: Thursday, April 09, 1998 3:16 PM Subject: RE: AI95-00085 >O_APPEND is a relatively new mode for Unix (speaking geologically here). >It was invented to solve the problem of having multiple processes >all writing to the same log file at the same time, wanting all messages >to end up at the end, no matter which process wrote them. OK. Sounds like my impression is correct: it is a junk mode created to patch around the lack of kernel-level sharing, good synchronization methods, and shared address space threads in [basic] Unix, and a lack of awareness of tasking issues in C. Such stuff is totally unnecessary from an Ada perspective, which has good synchronization methods and tasking. This just makes it a C-to-Ada interface issue, and such issues are best handled with forms and pragmas, not by messing with the basic Ada semantics. Randy. **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Thursday, April 09, 1998 4:10 PM Subject: Re: AI95-00085 | Of course you are allowed to silently close and reopen in response to | Set_Position (you can do it on every read if you like :-) The above would be very unwise in a UNIX system, since the semantics would be much different than if the file were to stay open. In particular, a file that is open cannot go away, even if it is (concurrently) unlinked, whereas once an unlinked file is closed it goes away. I suppose this is another respect in which the ARM semantics don't match UNIX semantics. The ARM says that when you Delete a file it ceases to exist. In UNIX, a file does not cease to exist until the last close. I can't imagine that existing Ada implementations on UNIX systems are required to somehow defeat the UNIX last-close semantics, so I guess the requirement about files ceasing to exist immediately after Delete is not being enforced. Given this, why not leave the Append semantics open enough that a direct mapping to O_APPEND mode is OK? --Ted **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Thursday, April 09, 1998 4:24 PM Subject: RE: AI95-00085 | I just don't understand why anyone would want to use O_APPEND ... | ... junk mode. ... I can't see any reason | that anyone would want the defined behavior of O_APPEND. ... O_APPEND serves a critical purpose, i.e., a safe way to write to log-files by multiple processes. When you use it, you DON'T want to allow positioning the file, and the mode prevents you from doing so; however, the file normally is disk file and so positionable if opened in other modes. [In case it is not sufficiently obvious, O_APPEND does something you *cannot* do with lseek() to the end of the file, because the lseek() and the following write operation are not atomic, and so one process could end up writing on top of something just previously logged by another.] | It was always my view that Append is a feature of Open/Create, not a mode. | It should have been implemented as an optional parameter to Open/Create. ?????????? Append *is* something you specify in Open/Create. --Ted **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Thursday, April 09, 1998 4:31 PM Subject: Re: AI95-00085 | Are you specifically talking about O_APPEND here? Certainly it should include O_APPEND, or more specifically the append-only (nonrepositionable, always-write-at the end) semantics. Otherwise, if the semantics could be emulated using other existing Ada operations, there would be no reason to add an append mode to the language. | We may say that the Ada 95 append mode was intended to allow access to | OS-provided append modes, but all OS's do not provide the same stuff, are | you saying that you think append mode should be entirely implementation | dependent? Not "entirely", but certainly loose enough to allow mapping to O_Append. --Ted **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Thursday, April 09, 1998 4:27 PM Subject: RE: AI95-00085 | However, A.14 makes it clear that each stream file object has its | own current_index, presuming the file supports positioning. | This would seem to preclude the use of O_APPEND, with its automatic | position-to-the-end semantics, unless positioning is disallowed | for the file. Hence it seems necessary to either raise Use_Error | on Set_Index, or not use O_APPEND mode for stream files. Presumably the only issue here is whether we want people to be able to use stream I/O to write log files from Ada programs in a concurrent environment. I've been presuming everyone would agree that the answer is "yes". --Ted **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 4:45 PM Subject: RE: AI95-00085 <> It is inappropriate to expect the Ada feature to provide this functionality. It is not documented to do so, and relying on it to do so, because you know your implementation happens to work this way is a style of non-portable programming that is not to be encouraged. If you want to rely on a Unix specific feature, use the Unix specific feature directly! <> No, it is a mode, and can be reset -- I very much agree that it should have been an optional parameter of Open/Create, but it is not! **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 4:42 PM Subject: Re: AI95-00085 <> Certainly GNAT does reopens in some cases, precisely to get a BETTER match to Unix semantics (i.e. do not open in update mode until you know you are going to be writing). <> Because it introduces unnecessary non-portability. If you think that it is fine not to be able to position, then your implementation can use O_APPEND *until* you reposition. If you want to use O_APPEND and reposition you are out of luck anyway! **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 4:46 PM Subject: RE: AI95-00085 <> Absolutely NOT! If you want to rely on Unix specific features for writing log files in a concurrent safe manner, then you should use the C streams and specific C/Unix features to do this. You can still use stream attributes in Ada to write and read the data. I see no particular merit in twisting Stream_IO into service for this purpose, since the resulting code would be annoyingly and obscurely implementation dependent. **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 4:52 PM Subject: RE: AI95-00085 <> What he means by junk (and I concur with his choice of the term) is some operating specific special feature meant to solve some problem in a peculiar operating system idiosyncratic feature. This feature may be interesting for Unix users, but it has nothing to do with Ada, and *allowing* (rather than insisting, which is impractical) implementations to map a specific restricted style of Ada stream_io (without a form parameter) to this strange Unix feature is not only not helpful it is actively harmful. If an implentor wants to provide this feature, the proper way to do it is with a form parameter, that specifies O_APPEND mode is to be used, and then of course you are free to enforce any restrictions you like. This is a perfectly valid approach, it makes the implementation dependence apparent, which is a GOOD THING, while preserving the capability you think is important in this specific environment. **************************************************************** From: Norman H Cohen[SMTP:ncohen@us.ibm.com] Sent: Thursday, April 09, 1998 4:52 PM Subject: Re: AI95-00085 It seems quite reasonable to assert that repositioning a file opened in append mode raises Use_Error, and to state that this happens portably and predictably for all implementations (Write Once, Blow Up Anywhere). This would allow Ada append mode to be directly mapped to Unix O_APPEND. More importantly, appending and repositioning are a logically inconsistent combination. -- Norman **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Thursday, April 09, 1998 4:58 PM Subject: Re: AI95-00085 <> It may be reasonable. It is a definite language change. It is perfectly reasonable to append, write stuff, go back and patch something you already wrote, position to the end, write some more, etc. so I disagree with the last statement. **************************************************************** From: Jean-Pierre Rosen[SMTP:rosen.adalog@wanadoo.fr] Sent: Friday, April 10, 1998 2:19 AM Subject: Re: AI95-00085 After reading the flood on this subject, I strongly concur with Robert's position, that I would rephrase this way: Those who want the semantics of a particular OS use the binding to this OS (and we do have a good binding to POSIX). Those who want portability across various OSes use Ada predefined services. Ted, are you discovering that portability sometimes requires the least common demoninator of various OSes ? ---------------------------------------------------------------------------- J-P. Rosen (Rosen.Adalog@wanadoo.fr) Visit Adalog's web site at http://perso.wanadoo.fr/adalog **************************************************************** From: Pascal Leroy[SMTP:phl@Rational.Com] Sent: Friday, April 10, 1998 5:46 AM Subject: Re: AI95-00085 Wow! 30+ messages in my mailbox this morning on this wretched AI! We have not had such a lively discussion on the ARG list for a long time. For the record, I agree with essentially everything that Ted said. > < append > mode raises Use_Error, and to state that this happens portably and > predictably > for all implementations (Write Once, Blow Up Anywhere). This would allow Ada > append mode to be directly mapped to Unix O_APPEND. More importantly, > appending and repositioning are a logically inconsistent combination. > >> > > It may be reasonable. I agree with Norm that appending and repositioning are probably inconsistent, and at the very least don't have a "natural" semantics. So if nothing else, this AI _must_ define what repositioning might mean for an append stream. > It is a definite language change. Definitely not. It is very much unclear what is the semantics of repositioning on an append file in the first place, and the RM doesn't give any clue as to what it should mean in Ada. (In fact, it's hard to find a section of the RM which is less precise than A.12.1.) So whatever we decide is not a language change, only a clarification. > It is perfectly reasonable to append, write stuff, go back and patch > something > you already wrote, position to the end, write some more, etc. so I disagree > with the last statement. That's one possible interpretation. Here is another one: On a stream opened in Append_Mode, Set_Index truncates the file at the specified position, and the next write will take place at the (new) end of the file. Surely this preserve two invariants: (1) Set_Index effectively moves to the specified position and (2) in Append_Mode, you only write to the end of the file. These two interpretations are (1) legitimate (2) not substantiated by the RM and (3) not contradicted by the RM. Pascal _____________________________________________________________________ Pascal Leroy +33.1.30.12.09.68 pleroy@rational.com +33.1.30.12.09.66 FAX **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Friday, April 10, 1998 6:03 AM Subject: Re: AI95-00085 Pascal says Definitely not. It is very much unclear what is the semantics of repositioning on an append file in the first place, and the RM doesn't give any clue as to what it should mean in Ada. (In fact, it's hard to find a section of the RM which is less precise than A.12.1.) So whatever we decide is not a language change, only a clarification. >> Robert replies: Well it is clear that positioning *is* allowed, so an AI that says it is not allowed is clearly a language change. Pascal suggests a possible interpretation of positioning: On a stream opened in Append_Mode, Set_Index truncates the file at the specified position, and the next write will take place at the (new) end of the file. Surely this preserve two invariants: (1) Set_Index effectively moves to the specified position and (2) in Append_Mode, you only write to the end of the file. These two interpretations are (1) legitimate (2) not substantiated by the RM and (3) not contradicted by the RM. Robert replies I disagree that the RM is unclear, and I do not think that Pascal's interpretation here is supportable. The RM simply says about Append_File that it causes the first transfer to be at the end of the file. I think what is going on here is that people read about append, and assume that for stream files it has something to do with O_APPEND in Unix, and then they start importing irrelevant semantics from Unix. When I looked at the RM, it was immediately clear to me that of course you can't use Unix append mode for stream_IO Append_File, since it has inconsistent semantics. We implemented a long time ago what seemed like the obviously required semantics, no one has ever complained, and no ACVC tests has contradicted this. I would be *very* reluctant to swich to an interpretation like the one above, which has no justification whatsoever in the RM, and might well cause silent malfunction of existing programs. Since Stream_IO is part of the core, there are 70 or so compilers that are implementing some interpretation or other of how this works. Can we please do a reasonably comprehensive survey of what implementors are in fact doing now, to at least make sure that whatever the ARG does in this non-critical area is not unnecesarily disruptive. I say non-critical here, because I have not seen users get worried about this issue! **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Friday, April 10, 1998 8:28 AM To: arg95@sw-eng.falls-church.va.us Subject: copy of Baker/Dewar exchange Robert Dewar has requested that I forward the following e-mail exchange between him and myself, which I intended as a private exchange, but Robert apparently intended for the whole ARG. My apologies for the length. --Ted --------------------------------------------- From baker Thu Apr 9 18:59:59 1998 To: dewar@gnat.com Subject: RE: AI95-00085 Content-Length: 556 It is unfair to cite me as a satisfied user of that GNAT idiosyncracy. My "use" of GNAT is primarily for implementing the POSIX.5 interfaces, and implementing the GNAT runtime. This feature does not come up there. That does not mean it is not an important functionality to real users. What really galls me about this discussion is that the reason for Append being added to Ada'9X was input from the POSIX Ada bindings group, and it specifically was intended to map O_APPEND --- despite whatever revisionist rhetoric you and Randy want to put out. --Ted From dewar@gnat.com Thu Apr 9 23:01 EST 1998 Return-Path: Received: from cs.fsu.edu by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4) id XAA20500; Thu, 9 Apr 1998 23:01:25 GMT Received: from sw-eng.falls-church.va.us (ns1.sw-eng.falls-church.va.us [199.75.54.2]) by cs.fsu.edu (8.8.8/8.7.3) with ESMTP id TAA22871 for ; Thu, 9 Apr 1998 19:01:27 -0400 (EDT) Received: from nile.gnat.com by sw-eng.falls-church.va.us (8.8.8/) id WAA11205; Thu, 9 Apr 1998 22:57:37 GMT Received: by nile.gnat.com (5.0/1.20) id AA28031; Thu, 9 Apr 98 18:59:04 EDT Date: Thu, 9 Apr 98 18:59:04 EDT From: dewar@gnat.com (Robert Dewar) Message-Id: <9804092259.AA28031@nile.gnat.com> To: Arg95@sw-eng.falls-church.va.us, baker@dad.cs.fsu.edu, dewar@gnat.com, ncohen@us.ibm.com Subject: Re: AI95-00085 Content-Type: text Content-Length: 393 Status: RO <> That's odd, the whole discussion is about *taking away* functionality from the current append semantics in the RM! From baker Thu Apr 9 19:15:45 1998 To: dewar@gnat.com Subject: Re: AI95-00085 Content-Length: 592 | That's odd, the whole discussion is about *taking away* functionality from | the current append semantics in the RM! The capability that is lost from UNIX is (atomic) writes to the end of a file by multiple processes. Providing that requires mapping directly to the OS O_APPEND mode, if there is one. If we require the Ada implementation to provide some other functionality (that can be simulated via other existing operations, and so is not a real gain) that required simulation of normal append mode writes via some non-atomic combination of system calls, then there is a loss. --Ted From dewar@gnat.com Thu Apr 9 23:05 EST 1998 Return-Path: Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4) id XAA20521; Thu, 9 Apr 1998 23:05:13 GMT Received: by nile.gnat.com (5.0/1.20) id AA02217; Thu, 9 Apr 98 19:03:03 EDT Date: Thu, 9 Apr 98 19:03:03 EDT From: dewar@gnat.com (Robert Dewar) Message-Id: <9804092303.AA02217@nile.gnat.com> To: baker@dad.cs.fsu.edu, dewar@gnat.com Subject: RE: AI95-00085 Content-Type: text Content-Length: 795 Status: R <<> This is not historically accurate, there were a number of revision requests for append mode, and they did not mention O_APPEND (I am not even sure it existed then!) What puzzles me Ted, is you had years to see a plain definition that was obviously inconcistent with O_APPEND semantics, and so did everyone else have this chance. The inconsistency was mentioned several times during the revision process. No one seemed concerned. No one raised the issue. It is a bit late to be suggesting significant changes to the language! From baker Thu Apr 9 19:19:28 1998 To: dewar@gnat.com Subject: RE: AI95-00085 Content-Length: 1006 | This is not historically accurate, there were a number of revision requests | for append mode, and they did not mention O_APPEND (I am not even sure it | existed then!) It certainly has existed, for many years, and has been in the POSIX spec. from the very beginning. The requests where I knew the people who put them in (like Dave Emery) were motivated by experience with UNIX/POSIX applications where the functionality was needed. Those requests may not have mentioned POSIX explicitly, so you may not have seen the connection. | What puzzles me Ted, is you had years to see a plain definition that was | obviously inconcistent with O_APPEND semantics, and so did everyone else | have this chance. The inconsistency was mentioned several times during | the revision process. No one seemed concerned. No one raised the issue. | It is a bit late to be suggesting significant changes to the language! Like several other things outside the tasking/AnnexD/AnnexC domain, I managed to miss this. --Ted From dewar@gnat.com Thu Apr 9 23:22 EST 1998 Return-Path: Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4) id XAA20558; Thu, 9 Apr 1998 23:22:20 GMT Received: by nile.gnat.com (5.0/1.20) id AA21075; Thu, 9 Apr 98 19:20:09 EDT Date: Thu, 9 Apr 98 19:20:09 EDT From: dewar@gnat.com (Robert Dewar) Message-Id: <9804092320.AA21075@nile.gnat.com> To: baker@dad.cs.fsu.edu, dewar@gnat.com Subject: Re: AI95-00085 Content-Type: text Content-Length: 1558 Status: RO <> Wait a minute, where is the loss here? There is nothing in the RM that talks about atomic writes by multiple processes. Such multiple writes to a file are clearly undefined in the Ada semantics. I really don't understand Ted's position at all. It seems to be something like: a) weaken the Ada semantics so that Ada compilers can use O_APPEND by default. b) do not require this weakening c) do not require the use of O_APPEND d) if you are lucky enough to have a compiler that takes advantage of this new functionality, then feel free to use it to write non-portable code. Seems quite strange to me. Ted, why do you object to a Form parameter something like Atomic_Append, which we all agree MUST have the semantics you want if it is implemented, and is rejected otherwise. Incidentally, I think Ted's history is totally wrong here. Append is seriously useful functionality for Sequential and Text_IO. Its provision in Stream_IO seems to have been mostly a matter of (possibly misguided) orthogonality. At least that is the way that Dewar remembers history! From baker Fri Apr 10 08:02:15 1998 To: dewar@gnat.com Subject: Re: AI95-00085 Content-Length: 2276 Sorry to spend so much talk on such a minor matter, but not allowing a mapping to O_APPEND still seems fundamentally wrong-headed to me, your repeated assertions notwithstanding. I guess we are not going to agree on this one. | Wait a minute, where is the loss here? Loss of permission to access to underlying OS functionality. | a) weaken the Ada semantics so that Ada compilers can use O_APPEND by | default. Weaken here means giving up a functionality (repositioning an append-only stream) that seems to have no practical use. | b) do not require this weakening Seems to mean the same thing as below. | c) do not require the use of O_APPEND In case the OS does not support O_APPEND. | d) if you are lucky enough to have a compiler that takes advantage of | this new functionality, then feel free to use it to write | non-portable code. I'd like to see this be the required semantics, for UNIX systems, e.g. by the POSIX Ada bindings. | Seems quite strange to me. Ted, why do you object to a Form parameter | something like Atomic_Append, which we all agree MUST have the semantics | you want if it is implemented, and is rejected otherwise. Does GNAT support such a form parameter? POSIX.5 defined an interface for this, before Ada'9X, but I have not seen the POSIX.5 form parameters supported. Moreover, last time we revised POSIX.5, people argued that we did not need this (complicated and unwieldy) form parameter business any more, given Ada'95 support for Append mode. In contrast to the form parameter, O_APPEND is very simple to implement (simpler than the form parameter, and simpler than trying to allow repositioning), and could easily become the defacto standard for systems that support it, if it were permitted. | Incidentally, I think Ted's history is totally wrong here. | Append is seriously useful functionality for Sequential and Text_IO. Its | provision in Stream_IO seems to have been mostly a matter of (possibly | misguided) orthogonality. | At least that is the way that Dewar remembers history! Yes, the addition to Stream_IO was clearly last-minute, and for orthogonality. The inclusion for Sequential and Text_IO is what I was taking about, where providing a mapping for O_APPEND is the historical basis. --Ted From dewar@gnat.com Fri Apr 10 12:20 EST 1998 Return-Path: Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4) id MAA20861; Fri, 10 Apr 1998 12:20:07 GMT Received: by nile.gnat.com (5.0/1.20) id AA22760; Fri, 10 Apr 98 08:17:54 EDT Date: Fri, 10 Apr 98 08:17:54 EDT From: dewar@gnat.com (Robert Dewar) Message-Id: <9804101217.AA22760@nile.gnat.com> To: baker@dad.cs.fsu.edu, dewar@gnat.com Subject: Re: AI95-00085 Content-Type: text Content-Length: 5379 Status: R <> Ah! But that is a TOTALLY different matter. Nothing we are talking about here requires any compiler to actually *support* what you want. If you want to go far beyond the AI, appropriate in the context of the POSIX Ada bindings, but clearly not in the context of the AI, and make such a requirement, then that's fine, but a Form parameter is a perfectly good (actually preferable) path to implementing this requirement. <> Too bad that people did not know what they were talking about! Clealry the Ada semantics as defined at the time did not support this argument. This is not an unknown issue, as my comments quoted from the GNAT sources show, I was aware of this issue years ago when I first implemented append mode. Indeed it seemed quite obvious to me that the RM was inconsitent with the use of the Unix append mode. I found it annoying, but there are many things an implementor finds annoying with the language! Actually I found it much more annoying that you don't know whether to open a file in read or update mode when you first open it. <> Certainly not! As I mentioned to you earlier, no customer of ours has ever indicated that this was of interest. <> No, that's wrong. The desire for append here has nothing necessarily to do with synchronization with multiple processes. It is a much simpler requirement, namely to be able to write starting at the end of the file. Recall that neither Sequential_IO nor Text_IO allow the possibility of positioning, so this fundamental capability, needed on all operating systems, and available on all operating systems, was missing from Ada 83. The revision requests in this area mentioned nothing about O_APPEND or the need for providing this unix/posix special synchronization mechanism. Indeed, NONE of the Ada 95 discussions raised the issue of this syncrhonization question. Perhaps it was always in the minds of the Posix folks who were involved in the discussion, and perhaps they always *knew* that this was the crucial issue, but if so, it is too bad that they (a) did not speak up and (b) let the language be designed in a manner inconsistent with the use of Unix append without comment. I have been involved in all the Ada 95 discussions, but not the Posix discussions. However, I would comment that I always thought that the basic motivation for the Posix append was the more general requirement. After all, the main use of the append mode in Unix is for the simple general requirement of doing appends. Even in Unix, the synchronization lockout use is somewhat specialized (at least that is my experience in seeing the mode used in real programs). By the way, what does the ANSI C standard have to say about appending exactly, can someone provide the quote. I think compatibility with C, as opposed to compatibility with Unix, always provides a stronger form of the compatibility argument. I certainly have no objection to the Ada/Posix interface requiring support of the Posix/Unix type append. However *even if* the AI is written to allow the default behavior to mirror the Posix way of doing append, I would be *really annoyed* if the Posix binding required you to change your compiler to implement things this way. Ted says <> That is completely incorrect. Implementing append by changing the current semantics, which our customers have been using for four years is not at all trivial. We will have to do a survey to see who would be affected, and most probably implement some complex mechanism to provide back compatibility to those who are affected. Implementing a form parameter is a simple job which should take only a day or two, and be completely upwards compatible. When the ARG changes the language in a manner that requires existing compilers to change their implementation, and existing users to modify their code, they are behaving in the most aggressive manner possible. This level of aggression is only approrpriate for absolutely major issues (we did even dare to do this for the 256 character case -- have indeed we EVER done this). I do not begin to believe that this issue is of sufficient importance to warrant a requirement that existing implementations change, and I strongly recommend that the POSIX group meet their (very reasonable) requirement by other means than requiring language and implementation chanes. Yes, we might have done things differently 5 years ago if the issue had been clearly understood, but since no one brought it to the table then, it is too late now. Ada 95 is not under construction! We are talking about features that have been available to users for four years, following a standard that has been available for over two years. From baker Fri Apr 10 08:05:59 1998 To: dewar@gnat.com Subject: RE: AI95-00085 Content-Length: 600 | I never saw an explicit request for Append mode for Stream_IO ... | Ted, can you give the exact reference, I dug a little, but could not | find anything at all. I'm sorry I don't have the original (foot-think) revision requests any more, due to lack of storage space. If you have the original revision requests documents on-line, and can't find it there, then I may have to partially concede that I remember it wrong. Of course my memory is weighted toward what was said at the requirements workshops (e.g. Soderfors and Ft. Walton Beach), and that is not written down anywhere to check. --Ted From dewar@gnat.com Fri Apr 10 12:29 EST 1998 Return-Path: Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4) id MAA20879; Fri, 10 Apr 1998 12:29:48 GMT Received: by nile.gnat.com (5.0/1.20) id AA22843; Fri, 10 Apr 98 08:27:35 EDT Date: Fri, 10 Apr 98 08:27:35 EDT From: dewar@gnat.com (Robert Dewar) Message-Id: <9804101227.AA22843@nile.gnat.com> To: baker@dad.cs.fsu.edu, dewar@gnat.com Subject: RE: AI95-00085 Content-Type: text Content-Length: 3415 Status: R <> I was at Soderfors and at Ft. Walton Beach, and I am 99% sure that the syncrhonization issue was never mentioned. I think what happened is that people just *assumed* that the Ada append mode would map into the Unix append mode, and so it was not discussed. I must say I made this same assumption. It was a surprise to find out that this was not the case, but I only noticed this later on when I came to implement it. Interestingly, in GNAT, we do not even use append mode for the Text_IO and Sequential_IO cases, because of orthogonality in the implementation for the Stream_IO case. That's probably undesirable, but actually until this discussion, I was completely unaware of the synchronization issue, and of course it is completely outside the Ada 95 semantics. Indeed, relying on Append_File mapping into the Unix mode append even for Text_IO or Sequential_IO is incorrect implementation dependent coding in any case, so it is not clear that a user should rely on it. In the GNAT documentation (I am not talking the sources now, but the user level documentation), things are quite clear: Table 6-1 Open and Create Call Modes OPEN CREATE Append_File "r+" "w+" In_File "r" "w+" Out_File (Direct_IO) "r+" "w" Out_File (all other cases) "w" "w" Inout_File "r+" "w+" If text file translation is required, then either b or t is added to the mode, depending on the setting of Text. Text file translation refers to the mapping of CR/LF sequences in an external file to LF characters internally. This mapping only occurs in DOS and DOS-like systems, and is not relevant to other systems. A special case occurs with Stream_IO. As shown in the above table, the file is initially opened in r or w mode for the In_File and Out_File cases. If a Set_Mode operation subsequently requires switching from reading to writing or vice-versa, then the file is reopened in r+ mode to permit the required operation. I think it is quite important to provide this kind of documentation. Note that it is quite clear from this that we do NOT use the a or a+ modes for Append_File, and as I said before, no user has ever questioned this decision. I can't say all our users read the documentation carefully, but I can say that a lot do. GNAT does provide a mechanism for opening a Text_IO or Sequential_IO file using a stream value obtained from a C library call (or C code making such a library call), and so this is the well defined mechanism in GNAT for making use of the Unix append modes (you will then of course get an exception if you try to do a positioning operation). I would assume that any of our customers needing the special synchronization mechanisms of Unix/Posix are using this approach, which seems quite appropriate, since it nicely encapsulates the implementation dependent part of the process. From baker Fri Apr 10 09:02:32 1998 To: dewar@gnat.com Subject: Re: AI95-00085 Content-Length: 771 | That is completely incorrect. Implementing append by changing the current | semantics, which our customers have been using for four years is not at | all trivial. We will have to do a survey to see who would be affected, | and most probably implement some complex mechanism to provide back | compatibility to those who are affected. ... | Ada 95 is not under construction! We are talking about features that have | been available to users for four years, following a standard that has been | available for over two years. I may have lost the thread by now, but I thought the AI came because an implementation was raising Use_Error in this situation, and that I've been trying to support the position that such an implementation be allowed to continue to do so. --Ted From baker Fri Apr 10 09:08:54 1998 To: dewar@gnat.com Subject: RE: AI95-00085 Content-Length: 610 | this discussion, I was completely unaware of the synchronization issue, | and of course it is completely outside the Ada 95 semantics. It is not exactly synchronization (between processes), but atomicity. All individual I/O operations are atomic up to some level of granularity defined by the OS (usually one disk block). The special issue here is atomicity of the combined seek-to-end + write. With append mode, you do get something you cannot do with the combination of lseek and write. In this regard, the UNIX kind of append mode also provides functionality beyond Ada's Direct_IO interfaces. --Ted From dewar@gnat.com Fri Apr 10 12:53 EST 1998 Return-Path: Received: from nile.gnat.com by dad.cs.fsu.edu with SMTP (SMI-8.6/SMI-SVR4) id MAA20939; Fri, 10 Apr 1998 12:53:39 GMT Received: by nile.gnat.com (5.0/1.20) id AA23135; Fri, 10 Apr 98 08:51:27 EDT Date: Fri, 10 Apr 98 08:51:27 EDT From: dewar@gnat.com (Robert Dewar) Message-Id: <9804101251.AA23135@nile.gnat.com> To: baker@dad.cs.fsu.edu Subject: RE: AI95-00085 Content-Type: text Content-Length: 338 Status: R Ted, is it really the case that my last few messages on this went only to you. If so that's really aggravating, since the whole point is to discuss this with the ARG. GRRRR! I see that is the case. Have you saved these messages, if so, PLEASE forward them to the whole group. It is really a waste of time for just the two of us to talk. From baker Fri Apr 10 09:12:14 1998 To: dewar@gnat.com Subject: Re: previous mail Content-Length: 213 I'll find the mail, and forward it. I hope it does not overwhelm everybody. I was getting uncomfortable about the volume and length of it, hence my limiting my reply to you only, a couple of cycles back. --Ted **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Friday, April 10, 1998 8:43 AM Subject: Re: last message of Baker/Dewar exchange | <> | Indeed, there is an implementation that wants to negate the standard by | raising Use_Error for all uses of positioning when Append_File is used. | We object because it is perfectly possible, even on Unix, and even if you | use the Unix append mode, to support positioning (by reopening the file). My disagreement here is that if you do this you hide a significant semantic event that the user did not expect (i.e., closing and then reopening) and which may have unwanted side-effects. If there were some important special Ada functionality here, this might be worth while, but no one has given any argument yet that explains how this functionality is needed. First of all, if repositioning is needed (even in other modes) with Stream_IO, why can we do without it for Text_IO and Sequential_IO? Second, if the semantics of repositioning in Append mode are the same as a combination of other operations that a user could have done, why do we need this mode at all? | But the suggestion that ALL implementations be required to raise Use_Error | (your suggestion) is way out of line. That is *not* what I'm suggesting. I'm suggesting that implementations be allowed (by the ARG) to raise Use_Error. What may have misled you is that I've also argued that other standards groups, like POSIX, be allowed (by the ARG) to narrow the semantic window for a particular OS. --Ted **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Friday, April 10, 1998 9:20 AM To: Arg95@sw-eng.falls-church.va.us; baker@dad.cs.fsu.edu Subject: Re: last message of Baker/Dewar exchange <> But these implicit reopens are part of life anyway, you have no way of avoiding these for the switch from reading to writing in a stream file, you certainly don't want to open all files in update mode just in case you may want to write, since that would mean you could not use stream_io on read only files. <> It is VERY useful to be able to reposition for stream files, code does this ALL the time. Sequential_IO and Text_IO are at a higher level where repositioning makes zero sense. <<| But the suggestion that ALL implementations be required to raise Use_Error | (your suggestion) is way out of line. That is *not* what I'm suggesting. I'm suggesting that implementations be allowed (by the ARG) to raise Use_Error. >> No, you are clearly suggesting this, *so that* you can then have the Posix binding require its use, so your ultimate agenda is precisely to require implementations to raise Use_Error. If this is not your intent (in the long run, including what you want the Posix interface to do), please clarify. To me, introducing implementation dependence here is a bad thing. Both in the case of direct Ada use of Stream_IO and the POSIX interface (which should support O_APPEND), I see no advantages in introducing implementation dependence. **************************************************************** From: David Emery[SMTP:emery@mitre.org] Sent: Friday, April 10, 1998 9:00 AM Subject: Re: last message of Baker/Dewar exchange Well, since my name was mentioned, I guess I'm entitled to contribute $.02 :-) Ted's recollection about 'intent' for Append_Mode matches my own. This was discussed within the POSIX.5 group, within the SIGAda ALIWG, and in some emails I remember back and forth during the Mapping and Revision process. The particular "desire" was to provide access to the functionality provided by Unix's O_APPEND for Ada files. There were two motivations for this: 1. Conformity with Unix semantics 2. It's generally useful. Within Unix, the 'formal semantics' mention atomic writes to the end of the file. At the user level, this degree of specification wasn't mentioned to my recollection. But the "semantics by analogy" from users was clear: We want Ada's Append_Mode to act like Unix's O_APPEND. I don't have my copy of POSIX.1 with me (I'm now working nearly full-time in the Pentagon :-(, so I can't look at the POSIX-specified for O_APPEND and lseek(). Robert Dewar argues, rightly, for surveying the vendors to see the degree of implementation approaches. At the same time, it is appropriate in an issue like this to survey users, too. Thus a message to comp.lang.ada and/or TeamAda that asks people about what they expect when they use Append_Mode would seem to me to be appropriate. Personally, I've never thought about the interactions between Append_Mode and file positioning, because my expected use for Append_Mode has always been for "write-only" files (e.g. logfiles), where file seeking makes no application-level sense. dave **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Friday, April 10, 1998 9:31 AM Subject: Re: last message of Baker/Dewar exchange <> We can get any answer we like here: What do you think Append_File should do? Most people will simply answer that it should start writing at the end of the file. Do you think Append_File should map to the Unix append mode? Most people will simply answer yes. <> So for you, the obvious implementation which Ted rejects would be fine. The issue is the following. Suppose you open a file in append mode and write some stuff Then suppose you do a positioning operation. Ted thinks it is fine to reject all positioning operations. Robert thinks it is easy enough to implement them, so you should. Ted doesn't like Robert's implementation because it involves closing and opening the file, he prefers to be able to completely forbid the positioning. The trouble is that the RM absolutely clearly and unambiguously, and intentionally, allows positioning and append_file mode to be combined. What Ted (and Dave perhaps) are recommending is that implementations be free to say "I could implement that, in a manner consistent with Ada semantics, but I don't think it's a good idea, and I would never use it, so I am not going to implement it." That's a heavy burden to support! **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Friday, April 10, 1998 9:34 AM Subject: Re: last message of Baker/Dewar exchange <> But appending only had to do with the initial position, as Ada is defined. Yes, you are certainly not appending in the Unix sense, but you know that, because the Unix sense of appending would not have allowed you to position. Actually I think one sense in which positioning and append are perfectly compatible from a conceptual point of view is if you are allowed to position, but only at or after the original append point. I hoped to be able to read this permission into the RM, since it seems to make a lot more sense, but in fact I concluded that it was absolutely clear that the RM requires you to be able to position behind the starting position, which I agree is bogus. I really think it would be useful to know what other compilers are currently doing, can we have some data on this please? **************************************************************** From: Robert I. Eachus[SMTP:eachus@mitre.org] Sent: Friday, April 10, 1998 10:40 AM Subject: Re: AI95-00085 At 09:30 AM 4/10/98 -0400, Robert Dewar wrote: > Indeed, there is an implementation that wants to negate the standard by > raising Use_Error for all uses of positioning when Append_File is used. > > We object because it is perfectly possible, even on Unix, and even if you > use the Unix append mode, to support positioning (by reopening the file). > > But the suggestion that ALL implementations be required to raise Use_Error > (your suggestion) is way out of line. Can we all agree to this? It allows both sides to go away happy. Yes, it does require a bit of extra work if the implementation chooses to make use of O_APPEND. And, yes, there are cases where the reopening can be detected or will fail. But it preserves the current RM semantics, and will allow implementations (including GNAT) to use O_APPEND if they choose to. A little more on what happens with the reopen. It is perfectly reasonable for an implementation (IMHO) to raise Use_Error in some cases (see A.12.1(33)) --repositioning on a pipe for example. But in general, an implementation must support repositioning on a stream connected to a "real" disk file, no matter what mode it was originally opened in. And there some are anomalies that should be left as implementation defined. (For example, if the file is deleted by one task while a second task is writing to the associated stream.) Robert I. Eachus with Standard_Disclaimer; use Standard_Disclaimer; function Message (Text: in Clever_Ideas) return Better_Ideas is... **************************************************************** From: Pascal Leroy[SMTP:phl@Rational.Com] Sent: Tuesday, April 14, 1998 6:15 AM Subject: Re: copy of Baker/Dewar exchange > By the way, what does the ANSI C standard have to say about appending > exactly, can someone provide the quote. I think compatibility with C, > as opposed to compatibility with Unix, always provides a stronger > form of the compatibility argument. The only interesting thing that I could find in the ANSI C RM is this: <<7.9.7.3 The fputc function If the file cannot support positioning requests, or if the stream was opened with append mode, the character is appended to the output stream.>> This is rather terse, but still much more precise than RM95 A.12. To summarize the ANSI C semantics is as follows: Positioning is supported for files opened in append mode (except of course for some special files, in an implementation-dependent manner). Positioning is in fact _useful_ if the file is opened in a+ or ab+ mode (reading and appending) because you can go back to some earlier location in the file, and start reading from there. However, each time you _write_ to the file, you go to the end. As already noted, it is rather unfortunate that Ada doesn't have read+write or read+append modes. If we wanted to mimic the C semantics, we would have to decide that Set_Index moves the index (and does not raise an exception); and that the next Write moves to the end of file no matter what. (I am not saying that we should do that, btw.) Pascal _____________________________________________________________________ Pascal Leroy +33.1.30.12.09.68 pleroy@rational.com +33.1.30.12.09.66 FAX **************************************************************** From: Robert Dewar[SMTP:dewar@gnat.com] Sent: Tuesday, April 14, 1998 5:24 AM Subject: Re: copy of Baker/Dewar exchange So one interesting conclusion from Pascal's summary of the ANSI C semantics is that forbidding all positioning for stream files in append mode, as suggested by some, would actually *decrease* compatibility with C, interesting ... Pity no one looked at the ANSI C standard more carefully if people think that compatibility with C is so important (I don't, my feeling is that if you want compatibiliy with C, please use the C routines directly, you can still us the stream attributes, so you don't lose any expressive power). **************************************************************** From: Ted Baker[SMTP:baker@dad.cs.fsu.edu] Sent: Tuesday, April 14, 1998 7:53 AM Subject: Re: copy of Baker/Dewar exchange | So one interesting conclusion from Pascal's summary of the ANSI C semantics | is that forbidding all positioning for stream files in append mode, as | suggested by some, would actually *decrease* compatibility with C, It would be compatible. Given that you can't read from an Ada file in append mode, the only use of such files is to write. In C repositioning in append mode has no effect on subsequent writes (they still go to the end of the file), you effectively cannot reposition. Of course, I agree that it would have been nicer if we had all sensible combinations of read, write, append in Ada, but we don't. | Pity no one looked at the ANSI C standard more carefully if people think | that compatibility with C is so important (I don't, my feeling is that | if you want compatibiliy with C, please use the C routines directly, | you can still us the stream attributes, so you don't lose any expressive | power). Admittedly, the only way to get true C compabibility. --Ted **************************************************************** From: Randy Brukardt Sent: Monday, January 15, 2001 5:41 PM I have received 16 test results for the AI-85 test so far. From these results, I can draw some preliminary conclusions. Then I'll go on to add my take on the AI as a whole. My first conclusion is that an ACATS test for Ada.Streams.Stream_IO is desperately needed. The testing uncovered bugs in the implementation of Stream_IO in 4 compilers, irrespective of the use of Append_File. Most of the problems have to do with the maintenance of the file index. Secondly, most compilers that target Unix do in fact use Append_File as meaning O_APPEND. However, none of the compilers come anywhere near the RM or in fact the current draft of the AI. Most compilers allow calls to Set_Index, then essentially ignore that those happened. Some even go so far as to return the Index value has having been changed by Set_Index, but then write at a different location in the file. ---- Technical Corrigendum 1 rewrote A.12.1 to clearly define the handling of the current file index in a positionable file to handle Defect Report 8652/0055 (AI-26). With those changes, the RM is much clearer on the handling of stream files. The current draft of AI-00085 is in fact a confirmation of the RM. Indeed, everything it says about Append_File mode stream files follows directly from the RM. Thus, in one sense, resolving this issue is easy. I see two problems with a strict reading of the RM. First, there is a hole in the RM in that it never says that Read and Write without index parameters actually Read or Write at the current index. (Compare A.12.1(30) to A.8.5(3).) Certainly, the intent of AI-26 was that the model of positionable stream files is similar to Direct_IO files. It certainly would be odd if that didn't include the location of Reading and Writing. Probably a sentence like the following needs to be added to the RM: "For a file that supports positioning, Read without a Positive_Count parameter starts reading at the current index, and Write without a Positive_Count parameter starts writing at the current index." I would assume the above is non-controversial, but since quite a few compilers ignore it, perhaps I'm wrong. Anyway, assuming the above, it is clear that a file opened with O_APPEND cannot support positioning. That follows since a Set_Index followed by a Write will not write at the current index, rather it will write at the end of the file. This is essentially what the test tries to do. This would cleanly provide the solution to the problem. However, (depending on how the RM is read), it may not be able to be used for a disk file. The problem is that A.12.1(2/1) clearly defines positioning as a property of the external file. However, the mode of the file (which is what O_APPEND is) is a property of the INTERNAL file. Thus, the mode of the file cannot affect its positionability. To stir up the pot some, this is the position that the test I distributed takes. It essentially tries writing the file in Append_File mode, then read the file in In_File mode. It then checks if the file is positionable. It then closes the file, reopens it, and again checks positionability. Note that this position essentially means that O_APPEND cannot be used to open disk files, or that the mode has to be silently changed if a call to Index or Set_Index is made. That's because any later (or earlier!) attempt to use the file must have the same positionability; and everyone agrees that we want positioning on Unix disk files opened in the In_File and Out_File modes. The question is how to handle this: 1) Do nothing, issue an ACATS test similar to the one I distributed. This would force Unix based implementations to limit the use of O_APPEND, and make its use semantically invisible. (Alternatively, they could provide access to it via a form parameter.) 2) Change the RM to allow positioning to be a property of the internal file. Then, an implementation could use O_APPEND for Append_File by making such files not positionable. An ACATS test similar to the one I distributed would be issued, with all of the "RLBCheck"s removed. (Append_File would only be lightly tested in this case). 3) Reject my argument above. In that case, no conclusions can be drawn from the RM on what files are and are not positionable. The effect and testing are similar to (2). Based on prior discussions, I'd expect that the number of people choosing (1) (for portability) and (2-3) (for capability) would be roughly equal. Perhaps some people have changed views? Personally, I don't see the point of insisting on Append_File mode being implemented with O_APPEND. The behavior of O_APPEND is rather unusual, and only applicable to a single OS. I would rather see a portable version of Stream_IO, with a form parameter to select the OS-specific behavior. This also would be more compatible with existing practice: all compilers tested allowed Index with Append_File; only one disallowed Set_Index with Append_File. If O_APPEND is deemed important, then clearly the file is not positionable, and Index and Set_Index raise Use_Error. This is less compatible, both for existing users (because they might be using Index), and for users porting from one system to another (code that would work on Windows would fail on Linux, for example). I think this sort of incompatibility needs a important reason to be triggered, and O_APPEND doesn't provide it. **************************************************************** From: Tucker Taft Sent: Monday, January 15, 2001 8:08 PM When we talk about the external file vs. the internal file, I believe we are talking about properties external to the Ada program vs. properties internal to the Ada program. There is nothing that says that a property of an external file can't change during its lifetime, as a side effect of what the Ada program does. Hence, if you use O_APPEND, then the external file is not positionable, because the external environment (i.e. the operating system) does not support positioning during appending. Hence, I don't think you can argue that the implementation is wrong if the positionability of the external file changes during its lifetime. For example, even though size is a property of the external file, it can change frequently. In fact, if something is a property of an external file, then there is not much you can say about it or how it changes. A.14 makes it clear that properties of an *internal* file are not affected by operations performed on other internal files associated with the same external file, but there is no real limitation on what can affect the properties of an external file. Given the above, I would say the positionability of a file is not guaranteed to stay the same forever, though of course it would be annoying if it changed capriciously, and presumably the market would weed out such implementations. The reason Unix provides the O_APPEND feature is that it is very common for multiple processes to share a file in Unix (e.g. a log file), and O_APPEND does the "right" thing in that environment. Other operating systems tend to be much less "friendly" to allowing multiple processes to have the same file open for writing, and hence the need for an O_APPEND is less critical. **************************************************************** From: Pascal Leroy Sent: Tuesday, January 16, 2001 2:58 AM > Personally, I don't see the point of insisting on Append_File mode being > implemented with O_APPEND. As Tuck pointed out, O_APPEND _is_ important to allow sharing of files to work correctly (e.g. log files, pipes, etc). Any change that would require an implementation to stop using O_APPEND would be an inacceptable incompatibility, since there is real code out there that depends on this capability. The alternative is to disallow positioning on an append file, and while it is an incompatibility too, it seems much less severe. > The behavior of O_APPEND is rather unusual, and > only applicable to a single OS. It is certainly not true that O_APPEND is "applicable to a single OS". I don't have an ANSI C manual handy, but open(2) is part of the standard libraries, and I wouldn't be surprised if O_APPEND were part of the standard. At any rate, OS vendors want C code to port easily to their OS, and so they better have some support for O_APPEND. Case in point: look for O_APPEND in the Visual C++ documentation and you'll find that it exists with exactly the same semantics as on Unix: **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 7:46 AM <<> Personally, I don't see the point of insisting on Append_File mode being > implemented with O_APPEND.>> It seems obvious that with the current language definition, Append_File cannot be implemented with O_APPEND. The following is from the GNAT sources: -- Note: we do not use "a" or "a+" for Append_File, since this would not -- work in the case of stream files, where even if in append file mode, -- you can reset to earlier points in the file. The caller must use the -- Append_Set routine to deal with the necessary positioning. Yes, that's unfortunate, but the design clearly does not permit this behavior. To "fix" this certainly requires a language change, and it seems very late to be making such a change to me. **************************************************************** From: Pascal Leroy Sent: Tuesday, January 16, 2001 8:00 AM > Yes, that's unfortunate, but the design clearly does not permit this > behavior. To "fix" this certainly requires a language change, and > it seems very late to be making such a change to me. Well, either way there is going to be some amount of incompatibility. Randy's experiments have proven that there are a number of compilers out there which do use O_APPEND, and we cannot ignore this fact. We could act as language lawyers, invent semantics for positioning on append files, and disregard the fact that there are probably users which depend on the O_APPEND behavior and hose code is going to break. However, that seems very wrong to me. At this late stage, we should only add incompatibilities if they buy us something. Sorry, I cannot get excited about positioning on append files, I am not even sure what this is supposed to mean in the first place. **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 2:04 PM Why do you think invention is needed here? THe RM seems clear to me. Once again, I think the only way the RM is puzzling is if you assume that append = O_APPEND. **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 2:01 PM <> I see no problem in the current definition, EXCEPT that it prevents the use of O_APPEND for append mode, but nothing in the RM says that you have to provide this capability. Since O_APPEND is inconsistent with Ada semantics, it should be controlled by a form parameter of some kind, we can certainly agree on the form parameter. I have not yet heard any specifics on how GNAT is not conforming to the RM, so that needs some report and investigation, but for sure I would assume that any compiler using O_APPEND cannot possibly be conforming. Furthermore, we can't make a non-backwards compatible change at this stage. Now of course if all compilers are significantly broken with respect to the current RM (note "all" and "significantly") then we have more flexibility in what we can do :-) **************************************************************** From: Tucker Taft Sent: Tuesday, January 16, 2001 2:50 PM Robert Dewar wrote: > Now of course if all compilers are significantly broken with respect to > the current RM (note "all" and "significantly") then we have more > flexibility in what we can do :-) I would agree that GNAT is conforming if it avoids O_APPEND and allows meaningful positioning in Append_File mode. However, you have never answered my point that it is likewise permissible for an implementation to make positionability change during the lifetime of a file. Of course, with GNAT going one way and Rational (and some others) going another way, we don't provide the commonality we are seeking. But I think either is a legitimate interpretation of the RM, since there is some much flexibility in how implementations handle "external" file characteristics. The next question is whether implementors are motivated to eliminate this implementation-dependence. I am hearing that Pascal and you don't agree about what interpretation is best, so that it might make sense to simply deem it implementation-defined whether Stream_IO files are positionable while in Append_File mode, thereby warning users of the possible non-portability. **************************************************************** From: Ada Rapporteur Group of WG9; managing the Ada issues [ARG@acm.org] on behalf of Robert Dewar [dewar@GNAT.COM] Sent: Tuesday, January 16, 2001 2:55 PM To: ARG@acm.org Subject: Re: AI-85 analysis and discussion. <> I don't understand this point, which is why I did not answer it. To me the RM requires that ordinary files in append mode be positionable. **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 2:56 PM I guess what surprises me here is that the issue has come up, it seemed obvious to me that you cannot use O_APPEND for append mode, and I never considered doing so. **************************************************************** From: Randy Brukardt Sent: Tuesday, January 16, 2001 4:18 PM My position is very simply: O_APPEND is an operating system specific facility. We have a mechanism for accessing operating system facilities in I/O: the Form parameter. Ergo, O_APPEND should be controlled by a form parameter. OTOH, I can't really understand Robert's objection. Robert said: "To me the RM requires that ordinary files in append mode be positionable." I don't see much in the RM that would support that position. The RM says (the second paragraph was added by the Corrigendum): 1 [The subprograms in the child package Streams.Stream_IO provide control over stream files. Access to a stream file is either sequential, via a call on Read or Write to transfer an array of stream elements, or positional (if supported by the implementation for the given file), by specifying a relative index for an element. Since a stream file can be converted to a Stream_Access value, calling stream-oriented attribute subprograms of different element types with the same Stream_Access value provides heterogeneous input-output.] See 13.13 for a general discussion of streams. Static Semantics 1.1/1 {8652/0055} The elements of a stream file are stream elements. If positioning is supported for the specified external file, a current index and current size are maintained for the file as described in A.8. If positioning is not supported, a current index is not maintained, and the current size is implementation defined. Nothing in this text makes any requirement on an implementation as to whether a particular file is positionable. Indeed, which files are positionable isn't even required to be documented, as it isn't even marked as implementation-defined. (Is that an oversight in the RM or was it really intended to be unspecified?) Thus, an implementation can make anything it wants non-positionable, as long as it properly implements the semantics of a non-positionable file. (Most implementations implementing O_APPEND do not do this currently, but there has been general agreement that this would be an acceptable resolution of the issue.) The Corrigendum text, however, does make it clear that positioning is a property of the external file (not the internal file). I argued that therefore whether or not a file is positionable cannot depend on the file mode (which clearly is a property of the internal file). Tucker counter-argued that the properties of an external file are those external to the Ada program, and that there is no requirement that they cannot change during the life of the file. If my argument is correct, then a implementation which treated a file opened with Append_File mode as non-positionable could not later treat the same file as positionable. This would be testable, and would effectively prevent using O_APPEND mode to open files (because you could not know when you were going to later open a file with another mode). If Tucker's argument is correct, then there is no problem with allowing the same file to later be opened (or reset to) another mode, and then allowing positioning in that mode. I have to admit, I find my own argument weak. It seems dubious to argue about requirements on unspecified stuff. However, if you reject that argument, then there is no problem whatsoever (vis-a-vis the RM) with opening files with O_APPEND, as long as such files are treated as not being positionable. In this case: -- The AI is a confirmation (because everything needed is already in the RM); -- An ACATS test to check that Append_File mode files are either positionable (and then positioning had better work right) or non-positionable should be issued, but this would have no effect on whether O_APPEND is used in opening files. We could consider adding some implementation advice on this issue. For instance, we could say that it is morally wrong to treat an ordinary disk file as non-positionable. (I certainly agree with that statement.) But from prior discussion, I think it is clear that we would never get consensus on such implementation advice. So I think it would be a waste of time to bother discussing that; I'd rather work on Tucker's Interfaces proposal :-). **************************************************************** From: Randy Brukardt Sent: Tuesday, January 16, 2001 4:33 PM > The next question is whether implementors are motivated to eliminate > this implementation-dependence. I am hearing that Pascal and you > don't agree about what interpretation is best, so that it might > make sense to simply deem it implementation-defined whether > Stream_IO files are positionable while in Append_File mode, > thereby warning users of the possible non-portability. This is a horrible idea. That would let us argue about this for many more years, and for what? Whether files are positionable in the first place is unspecified. Perhaps that should have been implementation-defined. To me, the only viable change to the RM would be to add a statement that "Which files support positioning, if any, is implementation-defined." That has nothing to do with Append_File; it just fixes an oversight in the RM. (I'd suggest fixing the Read without Index parameter goof as well, if we're making wording changes to this clause.) Let's not even think about getting append mode mixed up in it!!!! I think Implementation Advice saying "O_APPEND considered harmful." would also be appropriate, but I have no illusions that any such thing would ever pass. **************************************************************** From: Tucker Taft Sent: Tuesday, January 16, 2001 4:34 PM Robert Dewar wrote: > > < permissible for an implementation to make positionability > change during the lifetime of a file. Of course, with GNAT > >> > > I don't understand this point, which is why I did not answer it. > > To me the RM requires that ordinary files in append mode be positionable. The RM doesn't specify under what circumstances "positioning is supported for the [external] file." It says what should happen if positioning is or is not supported, but it doesn't say when or if positioning should be supported. So it seems permissible for positioning to be supported some of the time, and not at other times. In particular, it seems permissible for positioning to be disallowed while the file is being appended-to. **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 5:03 PM I disagree, if you allow this sort of thing, you legitimize any kind of bizarre behavior, and that is not the intention of the design. The only way positioning should not be supported here is if there is no reasonable implementation model. But there IS a reasonable implementation model for implementing append *as described in the RM*. If it was the intention of the design to allow O_APPEND as an implementation model here, then the design is indeed messed up. But we certainly have a reasonable implemenation model that provides the semanitcs in the RM. Therfore there is no good basis from an RM point of view of permitting an alternative implementation model that does not work. By the way, I agree it is *horrible* that the design disallows the use of O_APPEND, but that's the way the design is. You do not solve design problems by allowing implementations to arbitrarily ignore the semantic requirements. It is VERY late to be addressing this issue, certainly we can't disallow positioning of stream files. I guess at this stage, if we really want to allow the O_APPEND implementations to be considered conforming, we just bite the bullet and say that it is implementation defined whether positioning is allowed for stream files in append mode -- seems pretty ugly, but I see no alternative. We just *can't* go down Tuck's path here, it legitimizes absolutely any possible malfunction in the implementation of I/O. For example suppose we run Print_Line ("Hello"); and the output is Hello! and I note to the world that I have chosen an implementation model which does not support lines not ending in exclamation marks so I supply one by default (if you won't let me do that I will have to raise Use_Error). **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 5:09 PM <> I object to this, the only valid reason for not supporting positioning in the general case (forget about append) is if the underlying system does not support positioning. It is plain wrong for an Ada runtime to prevent positioning an arbitrary file. **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 5:13 PM <> I think this is perfectly deliberate AND reasonable. All files are positionable, there is no hint that this is not the case. On certain implementations, it may be the case that [stand by for RM 1.1.3(6) quote :-)] it is "impossible or impratical to avoid [the failure to support positioning] given the implementation's execution environment" for certain files, in which case, it is fine not to support it. But 1.1.3(6) is THE only justification for not supporting positioning, and weakening this any further is a very undesirable precedent. **************************************************************** From: Randy Brukardt Sent: Tuesday, January 16, 2001 5:31 PM To: ARG@ACM.ORG Subject: Re: AI-85 analysis and discussion. The words "(if supported by the implementation for the given file)" certainly imply to me that it is perfectly reasonable for a file to be non-positionable. If the intent was that all files must be positionable unless there is an insurmountable reason for them not to be, I would expect the wording to be very different. I certainly think it is a good idea for implementations to try to make as many files positionable as possible, but I certainly don't see any requirement to do so. **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 5:36 PM (if supported by the implementation for the given file) So it boils down to whether you think this is talking about whether the underlying file system supports this, or whether the implementor feels like doing it or not. For example, a liberal reading would allow an implementor to say that no files mentioned in the ACVC tests support positioning, but any other file does :-) (I would not approve this interpretation) **************************************************************** From: Randy Brukardt Sent: Tuesday, January 16, 2001 5:50 PM Honestly, I don't see a lot of difference. Tucker's argument is that the underlying file system does not support positioning for a file opened in O_APPEND, and certainly that is true. I personally think that using O_APPEND is a bad design decision by the implementor, but I don't think that the RM can do much to prevent bad design decisions. Certainly the RM would not object to an implementation that used 256-bit float numbers to do all integer calculations, but I certainly hope no one actually implements Ada that way. **************************************************************** From: Robert Dewar Sent: Tuesday, January 16, 2001 9:08 PM <> Bogus analogy! The choice of O_APPEND is not semantically neutral. The choice of 256-bit float for integer IS semantically neutral (omitting timing, which we never include in such considerations)(. **************************************************************** From: Randy Brukardt Sent: Tuesday, January 16, 2001 5:44 PM > But 1.1.3(6) is THE only justification for not supporting positioning, > and weakening this any further is a very undesirable precedent. The more I think about this, the more bizarre it seems to me. I don't know of any (other) place in the RM where it goes to substantial lengths to define what happens in a case that could only happen via a 1.1.3(6) argument. The usual idea is that such arguments are used to allow deviations from the standard. But in this case, we have a clearly defined concept (a file which does not support positioning). Such a concept would be necessary only if the standard anticipated a need for such files. But if such files are expected to exist, the standard certainly wouldn't then require all files to be positionable. (Or if it did intend to, it certainly would say so!) Now everyone knows that A.12.1 was horribly incomplete (just look at all of the stuff we had to add in the Corrigendum, and even then we didn't fix all of the problems). So the notion that whatever was meant to be said about supports positioning vs. does not support positioning was omitted is easy to believe. But I find your interpretation to be the least likely possible meaning. **************************************************************** From: Pascal Leroy Sent: Wednesday, January 17, 2001 3:26 AM > Why do you think invention is needed here? THe RM seems clear to me. Once > again, I think the only way the RM is puzzling is if you assume that > append = O_APPEND. I'm sorry, but ignoring Ada for a moment, in general computing terms, I don't understand what the interaction between appending and positioning is supposed to be. A.7(11) describes the various file modes and says "these values correspond to the cases where only reading, only writing, or only appending are to be performed." Note the phrase "only appending." To me, the verb append implies that the new data are added at the end. I am reading the following definition from the American Heritage Dictionary: "to add as a supplement or an appendix." So, if you are positioned somewhere in the middle of the file and you write, what is supposed to happen? Either you write where you are (and in this case do you truncate? do you overwrite?) and this is not "only appending." Or you go to the end and write from there (which is the O_APPEND semantics) and positioning is vacuous. Call it invention or call it clarification if you prefer, I believe we have to do something to explain the interaction between appending and positioning. > But we certainly have a reasonable implementation model that provides the > semantics in the RM. Therefore there is no good basis from an RM point of > view of permitting an alternative implementation model that does not work. Just out of curiosity, when "appending in the middle" of a file, do you truncate or do you overwrite? Both seem reasonable implementation models to me, and the RM doesn't appear to have a preference. > By the way, I agree it is *horrible* that the design disallows the use > of O_APPEND, but that's the way the design is. You do not solve design > problems by allowing implementations to arbitrarily ignore the semantic > requirements. The design is badly broken, in the sense that it mixes two abstractions that have nothing to do with each other. One is the append-only file, e.g. a socket, a pipe, a log file, etc. And the other is the file-as-an-array-of-bytes, with positioning/addressing capability. In isolations, these abstractions make perfect sense. But we have a problem when we try to divine what positioning/addressing means for an append-only file. So if the semantic requirements don't make any sense, well, ignoring them is probably a very reasonable option... > It is VERY late to be addressing this issue... I hate to mention this, but one of the reasons it is very late is that this AI has been dormant since October '98 because one Robert Dewar didn't do his homework ;-) > ... I guess at this stage, if we really want to > allow the O_APPEND implementations to be considered conforming, we just > bite the bullet and say that it is implementation defined whether > positioning is allowed for stream files in append mode -- seems pretty > ugly, but I see no alternative. I can live with that, and I believe it's close to Tuck's proposal. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 8:07 AM <> I don't see the relevance of the dictionary definition. The notion of appending is something defined by the RM, so we can only get information on what it means from there. If append was replaced by zork in the RM, this would not chnage the meaning of the concept. <> You overwrite, I don't see any expectation for truncation. Where in the RM do you get any implication that truncation would occur (it seems to me that your only reason for thinking this is reading semantics into the english word append that are not in the RM). It is a useful excercise here to indeed replace append with zork, and then reread the RM. I think if that is done, then things get clearer. I think it is clear that the truncation model is wrong, since A.8.2(9) pretty clearly says that the truncation model applies ONLY to sequential files. Pascal's view of append would correspond to compeltely disallowing positioning, as a matter not of implementation restriction, but from a fundamental logical point of view -- positioning incomaptible with appending, but the RM clearly allows positioning on append files, of that there is no doubt. This proves that the notion that Pascal has of append is not what the RM is talking about. There are actually three models that make conceptual sense: 1. Append has only to do with where you position the file at the end on open and reset, and otherwise is the same as Out (really should be In Out, but that's another story, the failure of stream io to have in out is to me also an annoying glitch). In this model, positioning is fully allowed, and you can position to a point BEFORE the original append point. 2. Append gives you a view of a file that consists only of the part you are adding, where you can position, but not before the original append point. To me that would make a whole lot more sense than 1, but there is no hint in the RM that positioning is restricted in this way, or that it is reasonable to restrict it. 3. Append does not allow positioning at all. Again, there is no hint in the RM that this is the intended meaning. All three models are well defined. I can only read the RM as permitting the first model, which to me is the least desirable of the three, I agree, so it would be fine to me to devise commonly accepted form parameters to specify either of the other two actions. One interesting data point here is that in GNAT we never use O_APPEND, not even for sequential files, because it is easier to treat all file types the same, and no one ever noticed, let alone asked for this to be changed, so this issue, though once on my radar screen when I first implemented things, since I was concerned about the consequences of the RM definition, long ago fell off the radar screen. What is the motivation for considering this as having higher priority now? Has some user actually run into problems, or is this just a matter of the old concern finally surfacing after years of neglect :-) **************************************************************** From: Pascal Leroy Sent: Wednesday, January 17, 2001 8:16 AM > I don't see the relevance of the dictionary definition. The notion of > appending is something defined by the RM, so we can only get information > on what it means from there. If append was replaced by zork in the RM, > this would not chnage the meaning of the concept. The word "append" is _not_ defined by the RM. Could you please tell me where you find such a definition? That word certainly doesn't appear in the index. (Well, the word Append, with a capital, appears in the index, but only because it's the name of some unrelated string subprograms.) When a term is not defined by the RM, 1.3(1) refers us to the Webster's Third New International Dictionary of the English Language. I don't happen to have this dictionary, but I draw two conclusions from this: (1) a dictionary definition is certainly relevant here and (2) your "zork" argument is irrelevant unless you can point to a definition of "zork" in the RM. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 8:39 AM <> THat's the point, it is an undefined term, replace it with Zork, and then you can only derive information from what's actually said about it, which is that on open and reset, the file is positioned at the end, there is NO other information, therefore the term has no other implication. The only reason you are building in other implications is that you think you know that the english word append should correspond to what the same english word happens to mean in some operating system. There is no implication in the RM that append has ANY effect on the ability to position a file, or that it has any effect other than affecting the initial position after an open or reset. The one clue that all is not right in the world is the "only appending" statement (let's remember that in the version we are examining, it says "only zorking"). Now if zorking has only to do with initial positioning, it is indeed a bit odd to consider that when you do subsequent writes, you are zorking (i.e. that the definition of zorking in this context is "writing after initially positioning the file at the end of the file, but intermediate positioning operations can have occurred). But following this thought would definitely lead to an expectation of a definite prohibition against positioning for append files, and this prohibition just is not there currently. **************************************************************** From: Ada Rapporteur Group of WG9; managing the Ada issues [ARG@ACM.ORG] on behalf of Pascal Leroy [pleroy@rational.com] Sent: Wednesday, January 17, 2001 8:56 AM To: ARG@ACM.ORG Subject: Re: AI-85 analysis and discussion. > THat's the point, it is an undefined term, replace it with Zork, and then > you can only derive information from what's actually said about it, which > is that on open and reset, the file is positioned at the end, there is NO > other information, therefore the term has no other implication. Hmm, using the same reasoning, I will point out that the word "end" is nowhere defined in the RM, and that you should really interpret the wording as saying that when a file is opened in zork mode, it is positioned at the zark after Open and Reset. Now of course you seem to be reading too much in the word "end", because you know what that term means in English, but there is no implication that "end" in the RM (which as we know is really zark) means the end of the file. Other than the fact that an RM where all undefined terms would be replaced by some z---rk word would be interesting reading, I find your reasoning absurd. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 9:12 AM <> Well perhaps you find it absurd, but this is precisely the way that the RM is written. Common english words like end are taken to have their normal meaning, but any technical term like append is quite different. To read into the term append a requirement that is nowhere in the RM is simply not valid in my view. I ask you once again, where in the RM do you find any hint or implication that append never allows positioning? If there was such an implication, why would it not be stated? To me, when I read the RM, it was immediately apparent that it was a little odd that append allowed general positioning. Actually, my concern was that I thought it made no sense to position BEHIND the append point, positioning after the append point makes perfect sense to me (and is consistent with your dictionary viewpoint, in the following sense, that opening a file in append mode, writing some stuff, with possible positioning as you go, and then closing it, results in appending stuff to the file in the dictionary sense). Indeed if you go with the dictionary definition, it definitely corresponds to this definition, and not to the UNIX viewpoint of not allowing positioning at all. Think of the following: a. please append your views to the end of the chapter. b. ok (starts scribbling, after a while, corrects a missing cross on a t in what was scribbled and continues) c. Hey! I told you to append your views, what the %$#@$%#@ are you doing going back and changing something you wrote b. Huh? -------------- Now, on the other hand, why does the implication that you can position BEFORE the append point worry me (that's the question I originally brought up), because now the dialog is: a. please append .... b. OK (starts scribbling, after a while goes and changes spelling in the original material) c. HEY! I told you to append your comments, not to go changing things d. Oh, sorry ... -------------- So I really don't think the dictionary definition justifies the no positioning viewpoint at all. Indeed it seems clear to me that if you want a no positioning regime, you should use a special FORM parameter, this is something different entirely than what the RM has in mind. The dictionary definition is sometimes useful in figuring out what someone had in mind, but it does not affect the definition in a case like this. You are doing something different here, you are not trying to connect append in the RM to append in the dictionary, but rather append in the RM to O_APPEND in Unix. There is simply no justification for this attempted connection in RM terms. **************************************************************** From: Pascal Leroy Sent: Wednesday, January 17, 2001 9:25 AM > You are doing something different here, you are not trying to connect > append in the RM to append in the dictionary, but rather append in the RM > to O_APPEND in Unix. There is simply no justification for this attempted > connection in RM terms. No, that's an unfair comment. I am not trying to connect anything to O_APPEND. I am trying to understand what is the meaning of positioning for an append file, based on what the RM says and based on the usual meaning of English words. This seems to be a prerequisite to taking any decision on this issue. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 9:44 AM OK, here is my position. 1. Based on what the RM says, it seems clear that general positioning is allowed, because it is not disallowed. 2. Based on the normal english meaning of the word, appending means adding something to the file, and excludes modifications to the existing part of the file. This means that positioning for modification should be restricted to the appended section. I can't see any justification, EITHER in the RM OR in the dictionary for forbidding positioning completely. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 10:06 AM Robert Dewar wrote: > ... I guess at this stage, if we really want to > allow the O_APPEND implementations to be considered conforming, we just > bite the bullet and say that it is implementation defined whether > positioning is allowed for stream files in append mode -- seems pretty > ugly, but I see no alternative. We should be careful here to allow what to me is the most natural expectation, namely that positioning is allowed, but the position is relative to the part you are adding. **************************************************************** From: Tucker Taft Sent: Wednesday, January 17, 2001 10:02 AM Robert Dewar wrote: > ... I guess at this stage, if we really want to > allow the O_APPEND implementations to be considered conforming, we just > bite the bullet and say that it is implementation defined whether > positioning is allowed for stream files in append mode -- seems pretty > ugly, but I see no alternative. This is the compromise that Pascal and I are also supporting. I believe there are ample "legalisms" to justify our position, but I also hate the idea of implementors generally using legalisms to support all kinds of bizarre behavior, so I see Robert's point as well. I have some memories of why we added Append_File mode as related to being able to use Unix O_APPEND, because it has nice properties in the presence of file sharing. Otherwise, why go to the trouble of having Append_File mode, since you can just do a Set_Index to the size of the file + 1? In any case, at this point I recommend we adopt the compromise suggested above by Robert, and give up on the more philosophical discussion. **************************************************************** From: Randy Brukardt Sent: Wednesday, January 17, 2001 2:15 PM Tuck said: > I have some memories of why we added Append_File mode as related > to being able to use Unix O_APPEND, because it has nice properties in > the presence of file sharing. Otherwise, why go to the trouble of > having Append_File mode, since you can just do a Set_Index to the > size of the file + 1? If the language design really had done this, I would have fought it tooth-and-nail. But the reason we have Append_File mode is so that Sequential_IO and Text_IO files can be appended on. They don't have an alternative way to get the end, and this was a significant problem. I certainly did not know about the odd semantics of the Unix Append mode, and I certainly would not have supported anything that would include that semantics in the Ada standard. Why we have an Append_File on stream files I do not know. I think it was misguided orthogonality. (It's clear that no one reviewed the Stream_IO section carefully, given how unclear and incomplete it is.) The only possible use for such a mode would be on a file that does not support positioning. (Note that I always thought that the semantics of Append_File were what Robert describes as option 1: "Append has only to do with where you position the file at the end on open and reset, and otherwise is the same as Out." I'm quite amazed that anyone would think otherwise -- which I think is the reason this debate is so hard to resolve. Since the RM doesn't say much about Append_File mode, everyone seems to think it matches whatever their expectation is; and those are all different.) Back to Tuck: > In any case, at this point I recommend we adopt the compromise > suggested above by Robert, and give up on the more philosophical > discussion. I don't have a problem with the compromise, except that it appears to be completely unnecessary. The only real question is whether the RM requires files to be positionable. If the answer is no (which I believe is the only reasonable position), then the RM as it stands is flexible enough to handle any problems. If the answer is yes (Robert's position), then we need not only Robert's compromise, but also some statement to the effect that files are supposed to support positioning unless impossible or impractical. There is no such implication currently in the RM. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 7:13 PM <> Ah well, we have QUITE different reactions then, because to me it is clear that Append is very useful, and SHOULD allow positioning, but should NOT allow positioning before the append point. That's what I would expect from *my* dictionary definition :-) **************************************************************** From: Randy Brukardt Sent: Wednesday, January 17, 2001 8:12 PM Yes, that is becoming clear to me. Of your previously mentioned three possible semantics, I think #1 was meant, you think #2 was meant, and Pascal (based on his previous mail) seems to think #3 was meant. This explains why it is so hard to get agreement on this issue! It's unfortunate that neither the RM nor AARM give any inkling as to what the designers of Ada 95 had in mind. Thus we all guess, and apparently we guess differently, based on our experiences. While it might be valuable to tie this down, I suspect that reaching consensus on it could be very difficult. I simply don't think it is worth the time, and given that the RM doesn't seem to outright prohibit what some implementors want, I think I would prefer to simply say nothing (beyond noting in the AI that some implementations may choose to make Append_File mode files non-positionable in order to use O_APPEND). This has the advantage of making the AI a confirmation, and would allow us to test it soon (based on my recent tests, I think that would benefit Ada users by improving the likelihood that Stream_IO is implemented correctly). **************************************************************** From: Randy Brukardt Sent: Wednesday, January 17, 2001 8:27 PM To make it clear how I propose to resolve this AI: I would leave the AI as a confirmation. I would add some text to the discussion section to make the resolution clear, that is that we're adopted Robert's compromise. That is, "It is implementation defined whether positioning is allowed for stream files in append mode." Saying so in the discussion would be intended to put up a flag for users. I'd also figure to add that as an annotation to the AARM the next time it is revised. I do agree that the AI should mention it. I do not agree that the RM needs wording changes to mention this; it follows from the fact that it is unspecified whether positioning is allowed for a particular stream file. The RM does need a wording change to say the obvious: that Read (and Write) without an index parameter read (or write) at the current index if the file supports positioning. I don't think we need to clutter this AI (which also contains three other questions, all confirmations) with that wording. Personally, I think that the RM should also have said that "It is implementation defined whether positioning is allowed for a particular stream file." (rather than leaving it unspecified), so that users can verify that their implementations are not playing games. It is unfortunate that this was left unspecified in the RM, because it does open the door to all kinds of games. But I think that the market is better at eliminating these sorts of games rather than the RM and ARG and ACATS. If we want to consider either of these wording changes, I think it would be best to give them their own AI (separate from the other questions of AI-85). And I think that AI should get a fairly low priority. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 7:20 PM <> I disagree, the ACATS tests should test positioning on normal files, and any implementation not providing this behavior should be considered to fail these tests. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 10:05 PM I must say I cannot understand Randy's thinking that interpretation one was meant (though of course, I agree that's what is said). How can you possibly regard a sequence of actions that rewrites an existing file as being doing "only append" operations. **************************************************************** From: Randy Brukardt Sent: Wednesday, January 17, 2001 10:26 PM > I must say I cannot understand Randy's thinking that interpretation one > was meant (though of course, I agree that's what is said). How can > you possibly regard a sequence of actions that rewrites an existing > file as being doing "only append" operations. I wasn't aware of the statement Pascal dug up until this morning (A.7(11)). My understanding of Append_File goes back a lot further than that. Besides, I do not believe that there is any value to an "only appending" mode for stream files (because, like you, I believe virtually all such files should be positionable). So I view that wording as yet another unfortunate oversight in a pile of them for Stream_IO. Finally, if you take that literally, then positioning on Append_File mode files must be banned. (That is, interpretation #3.) I don't think we can afford to take that position: not a single compiler tested to date actually treated Append_File mode files as not supporting positioning. Insisting on that now would be a massive incompatibility. **************************************************************** From: Jean-Pierre Rosen Sent: Thursday, January 18, 2001 11:09 AM > > I must say I cannot understand Randy's thinking that interpretation one > > was meant (though of course, I agree that's what is said). How can > > you possibly regard a sequence of actions that rewrites an existing > > file as being doing "only append" operations. > > I wasn't aware of the statement Pascal dug up until this morning (A.7(11)). > My understanding of Append_File goes back a lot further than that. > Following this thread, I reread that paragraph. Nobody mentionned it, so I just add as a piece of information that direct files do *not* have an append mode. This supports the idea that direct positionning and Append_Mode are somehow incompatible. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 10:06 PM <> Let me repeat again that this wording is NOT acceptable, since it implies only intepretations 1 and 3, and does not permit 2. **************************************************************** From: Randy Brukardt Sent: Wednesday, January 17, 2001 10:20 PM I can't imagine any (simple) wording which would allow 2. Doing so would require an entire new set of semantics for Index/Set_Index etc. Such a "partially positionable" file would certainly need changes in a bunch of paragraphs of the standard. I don't think this problem is anywhere important enough to justify any language change at all, much less a wholesale one. **************************************************************** From: Robert Dewar Sent: Wednesday, January 17, 2001 11:20 PM Maybe not, maybe the external file is only the appended part in this case, after all the RM does not define what a file is. This actually seems a perfectly valid semantics. **************************************************************** From: Ted Baker Sent: Thursday, January 18, 2001 6:48 AM | <> -- Randy | Let me repeat again that this wording is NOT acceptable, since it implies | only intepretations 1 and 3, and does not permit 2. --Robert It is an essential feature of append mode that the file CANNOT be repositioned, or to be more precise, it cannot stay repositioned if repositions, since every write operation is required to append to the end of the file. That is an invariant on which applications will rely. **************************************************************** From: Robert Dewar Sent: Thursday, January 18, 2001 8:08 AM <> Not append mode as defined in the RM. Please justify such sweeping statements by reference to the RM. <> Not if they are reading the RM! **************************************************************** From: Ted Baker Sent: Thursday, January 18, 2001 5:25 AM | > I have some memories of why we added Append_File mode as related | > to being able to use Unix O_APPEND, because it has nice properties in | > the presence of file sharing. Otherwise, why go to the trouble of | > having Append_File mode, since you can just do a Set_Index to the | > size of the file + 1? | If the language design really had done this, I would have fought it | tooth-and-nail. | ... I always thought that the semantics of Append_File were what | Robert describes as option 1: "Append has only to do with where you position | the file at the end on open and reset, and otherwise is the same as Out." | I'm quite amazed that anyone would think otherwise ... Sorry, Randy, but Tucker is right. I was there, and remember being one of the ones who made the argument, remember that this view was voiced, clearly. It may be that you and some others heard what you wanted to hear, though. The members of the POSIX Ada bindings working group brought this up. Unix/POSIX systems require that for this mode "the file offset shall be set to the end of the file prior to EACH write". On the "better" implementations, this is done atomically, so that multiple processes may log to the same file concurrently, without fear of losing any data due to overwriting. However, there seems to be some controversy within the Unix standards group over whether atomicity is required, since the above-quoted phrase not clearly state that. I hope you will realize that atomicity of writing and positioning to the end here offers a distinct benefit, just as does atomicity of checking for prior existence of a file and creating a new file of a given name. Because of these benefits, these features are provided in atomic form on most modern operating systems. Our goal in getting an append mode into Ada 95 was to not deny access to these valuable features to Ada language users. **************************************************************** From: Robert Dewar Sent: Thursday, January 18, 2001 8:02 AM <> Maybe, but there is no trace of this requirement in the RM. <> Maybe, but there is no trace of this feature in the RM <> Maybe, but then "you" [whoever the you is] messed up. In particular, consider the following 10 Severs the association between the given file and its associated external file. The given file is left closed. In addition, for sequential files, if the file being closed has mode Out_File or Append_File, then the last element written since the most recent open or reset is the last element that can be read from the file. If no why are stream files excluded here? I do not contest that it would be useful to have the mode you suggest, I just do not see that there is any trace of intent to have this mode for *STREAM* files in the RM, though of course sequential files do indeed work the way you mention (is your memory of what happened perhaps related to sequential files). I say this because IF there was ever any clear discussion of the intent here, then what is in the RM is even more decisive, because it means it was written with a clear awareness of the issue. So far, at least some people in this discussion have been assuming that the RM is "wrong" in this area as a result of NOT properly considering the issue of stream files and append. If indeed as Ted remembers, it was carefully considered, then one would be more inclined to think that the design in the RM is indeed deliberate, and in particular that the exclusion of stream files in para 10 above was deliberate. **************************************************************** From: Pascal Leroy Sent: Thursday, January 18, 2001 8:33 AM > Maybe, but then "you" [whoever the you is] messed up. In particular, > consider the following > > 10 Severs the association between the given file and its associated > external file. The given file is left closed. In addition, for > sequential files, if the file being closed has mode Out_File or > Append_File, then the last element written since the most recent open > or reset is the last element that can be read from the file. If no > > why are stream files excluded here? It is interesting to note that AARM A.8(2.a-b) explains that the authors of the RM considered defining 'sequential files' to mean Sequential_IO and Stream_IO files. There is a possibility that the omission of stream files from the above paragraph is a historical accident resulting from a shifting definition of 'sequential file'. Of course, there is no way to know at this point, and any reasoning based on reconstructing the intent is weak. However, we must not ignore the fact that the text of the RM as we read it today might merely be the result of incomplete/incorrect editing. **************************************************************** From: Robert Dewar Sent: Thursday, January 18, 2001 8:44 AM <> An interesting thought indeed. Reacting to Ted's message, if he insists that applications will rely on the meaning of append as he sees it, and if (as we know) some but not other implementations provide the semantics he thinks applications will rely on, then we have a significant mess on our hands. What we don't know is the following: 1. For implementations that use O_APPEND and do not allow positioning (Rational, ....) How many applications really are relying on this semantics. 2. For implementations that allow positioning (GNAT, ...) how many applications actually do the positioning operations in append mode. Now let's assume both these sets are non-empty. Clearly applications in category 1 are worrisome because a) they will be blown up in non-repeatable non-obvious ways by either an ARG ruling requiring positioning, or moving to a positioning compiler. Furthermore, if such a change was made by the ARG, then the fix (switch to sequential io) is painful (and may not even work on compilers not supporting the necessary file sharing semantics). If applications actually DO positioning operations in append mode, then if the positioning were disallowed, the situation is not so bad. a) they get a clear use error with a message saying what is going on b) the fix is simply to reset into write mode before the positioning operation (note that for implementations allowing my first model of append, the one Randy thinks is most natural, and the one I think that the RM specifies now, append mode is equivalent in any case to just opening in write mode and positioning to the end manually). This makes it clear that we really cannot "fix" this situation by requiring model 1 semantics, even if Robert and Randy are quite sure that's what the RM says. That is, pragmatically, if the set of users who are depending on O_APPEND is non-zero (and O_APPEND compilers have no way of knowing if this is a non-empty set), then implementors of O_APPEND compilers are simply NOT going to change their implementations just to satisfy some ARG ruling that does not affect any of their current customers except negatively. On the other hand, non O_APPEND compilers can likely change to O_APPEND mode with less disruption, so that's not out of the question to mandate. If Ted is really right, and there are people who are depending on this, even if they can't justify such action from the RM, then maybe it is a mistake to leave this implementation dependent. At the very least I would suggest no action, rather than making it implementation dependent, since a specific action of making it ID does not improve the situation in my view. **************************************************************** From: Ted Baker Sent: Thursday, January 18, 2001 4:27 PM | 1. For implementations that use O_APPEND and do not allow positioning | (Rational, ....) How many applications really are relying on this | semantics. As noted in my later e-mail, the POSIX semantics seem to allow nominal repositioning, so that one might interrogate the position (or do a read) and find that the position is not at the end of the file, but that would have no effect on the output. It would still atomically reposition to the end of the file each time a write is done. I don't recall ever hearing that anybody relied on this, though. I believe the main use of O_APPEND is for safe logging. **************************************************************** From: Ted Baker Sent: Thursday, January 18, 2001 4:12 PM | ... is your memory of what happened perhaps | related to sequential files .. Possibly. In the POSIX context we were talking about operations on "files", not streams. I believe the original user input asking for an append mode dated back into the Ada 83 days and was based on the idea of adding a new mode to the Ada 83 I/O model. Streams were not around in Ada yet. My belief is that the two ideas met later in the Ada 9X draft, without a lot of discussion of the interactions. BTW, it is interesting to note that the subject of O_APPEND has led to interesting interpretation discussions in the POSIX domain as well. The question came up in the Austin Group discussions about what happens when one repositions a file that was opened with O_APPEND, e.g., using lseek(). The answer seems to be that you can change the "current position" between writes (i.e., between appends), but when you do a write that position has no effect (i.e., the writes still append to the end of the file). --Ted Excerpts from Austin Group e-mail on this subject: ... the requirements stated for open() (with or without the O_APPEND flag) is that the file offset for the new file descriptor is set to be 0 (see XSH6d4, P1351, L27378-27379). The file offset doesn't move to the end of the file until one of the functions that explicitly moves the file offset or writes data to that file descriptor completes successfully. There is no requirement that the file offset always be set to the end of file on a file opened with O_APPEND; only that all writes add data to the end of the file rather than overwriting data already present in the file. **************************************************************** From: Randy Brukardt Sent: Thursday, January 18, 2001 4:07 PM > | If the language design really had done this, I would have fought it > | tooth-and-nail. > | ... I always thought that the semantics of Append_File were what > | Robert describes as option 1: "Append has only to do with where you > | position > | the file at the end on open and reset, and otherwise is > | the same as Out." > | I'm quite amazed that anyone would think otherwise ... > > Sorry, Randy, but Tucker is right. I was there, and remember > being one of the ones who made the argument, remember that this > view was voiced, clearly. It may be that you and some others > heard what you wanted to hear, though. I'm afraid you failed to read my statement carefully. I specifically said "If the LANGUAGE DESIGN had done this...". I was well aware that some POSIX-centric lobbying existed for Append_File, and I did my best to prevent any OS-specific features from getting added to Ada 95. My view of append is the only one that could make sense on all of the weird OSes to which Ada has been ported (from DOS to POSIX, from VMS to OS-1100), and I made sure that the RM language did not require anything beyond that view. If there had been such language, I would have objected. (Of course, like everyone else, I never gave Stream_IO a careful reading...) > On the "better" implementations, this is done atomically, so that > multiple processes may log to the same file concurrently, without > fear of losing any data due to overwriting. However, there seems > to be some controversy within the Unix standards group over > whether atomicity is required, since the above-quoted phrase not > clearly state that. > > I hope you will realize that atomicity of writing and > positioning to the end here offers a distinct benefit, just > as does atomicity of checking for prior existence of a file > and creating a new file of a given name. There probably is a benefit of atomicity for low-level writing. But that benefit does not extend to Stream_IO. That's because there is no limit on the number of calls to the stream Write that a Write or Output attribute can make (with the possible exception of an elementary type attribute). An Ada implementation would have to go to heroic measures (*) in order to guarantee that, and there certainly is no requirement in the RM to do so. Thus, any program that depended on such atomicity is completely implementation and target dependent. For instance, consider two Ada programs, both using O_APPEND to write to a log file. Each log entry is a record including a string message, a user id, and an time stamp. The canonical semantics for 'Write would involve a call to the stream Write for each component, including each individual character of the message! An compiler could use fewer calls than this, but that is only possible in isolated circumstances (no user defined attributes, no representation clauses) and in any event is not guaranteed. If both programs started writing a log item at the same time, the OS atomicity would only guarantee that a particular call to Stream_IO.Write did not overlap another. Thus, it is very likely that the two records would end up interspersed, and they could not be read by any means at all. The same problems could happen for Text_IO, but there the result is much less likely to be fatal. Thus, I conclude that atomicity of O_APPEND is not useful for Stream_IO files (at least, not unless there are major changes to other parts of the stream system). The other feature of O_APPEND (automatic seeking to the end) is actually harmful for an stream file, as it increases the chance that what is written cannot be read back. Thus, any effort to support O_APPEND on stream files is wasted. (This argument may not apply to Text_IO files.) Randy. (*) A implementation that did want to guarantee atomicity of a particular Write would have to do the following: -- Define additional atomicity operations in the private part of Root_Stream_Type, with null implementations; -- Override these operations in the private part of Ada.Streams.Stream_IO; -- Generate calls to these operations at the start and end of each top-level stream attribute call; -- Include a buffer and lock in each file object; -- For a start_atomic, initialize a buffer, and lock a lock; -- For each following Write, check that the correct task is doing the operation (if not block it until end_atomic); write the contents into the buffer. -- For end_atomic, write the contents of the buffer to the OS and unlock the lock. This requires the ability to buffer an unknown number of stream elements per operation, and thus requires some sort of dynamic allocation of buffers. The lock could be omitted if the implementation is willing to make concurrent access from the same partition erroneous (it seems very strange to insist on atomicity between *different* partitions, but not care for the *same* partition). Even with this scheme, a user-defined stream could not take advantage of these facilities (because they're not part of the public interface of Root_Stream_Type). Thus, users would be denied the opportunity to use this information in their own streams. (Claw, for instance, declares a stream type in order to be able to allow arbitrary Ada objects to be written to the Windows Clipboard. The Clipboard requires all of the data to be presented at once, thus we need a buffering stream much like the one described above.) Clearly, the RM does not require all of this buffering and extra stream calls, and any programmer depending on its existence is implementation-dependent in the worst way. **************************************************************** From: Randy Brukardt Sent: Thursday, January 18, 2001 4:22 PM Robert wrote: > What we don't know is the following: > > 1. For implementations that use O_APPEND and do not allow positioning > (Rational, ....) How many applications really are relying on this > semantics. > > 2. For implementations that allow positioning (GNAT, ...) how many > applications actually do the positioning operations in append mode. This is not a correct view of the actual situation. I realize that I haven't distributed the test results yet, because results are still trickling in. 1. For implementations that use O_APPEND and do not allow positioning How many applications really are relying on this semantics. The OcSystems compiler is the only (fielded) one that disallows the use of Set_Index on a Append_File mode stream file. Tucker changed his implementation during the early discussion, but I doubt any users have seen that yet. 2. For implementations that allow positioning (GNAT, ...) how many applications actually do the positioning operations in append mode. OK, but there is a third possibility: 3. For implementations that use O_APPEND and allow positioning (but it still writes at the end) (Rational, Green Hills, probably Aonix). How many applications really are relying on this semantics. How many applications use Append, rely on the seek to end semantics and also rely on Index (which would be disallowed if positioning is). Note that the last set might be empty, as the Index in some of the compilers at least returns the value of Index as if the Ada semantics were followed, as opposed to the correct value (where you will write). > Now let's assume both these sets are non-empty. Clearly applications > in category 1 are worrisome because a) they will be blown up in > non-repeatable non-obvious ways by either an ARG ruling requiring > positioning, or moving to a positioning compiler. Furthermore, if > such a change was made by the ARG, then the fix (switch to sequential > io) is painful (and may not even work on compilers not supporting the > necessary file sharing semantics). I think that the set of applications successfully in category 1 or 3a is probably null, because of the reasons mentioned in my response to Ted. OTOH, there may be applications in these categories that *think* they work, so the impact may be non-zero. > If applications actually DO positioning operations in append mode, > then if the positioning were disallowed, the situation is not so bad. > > a) they get a clear use error with a message saying what is going on Given the quality of exception handling in many Ada compilers, I would not depend on the message part. > b) the fix is simply to reset into write mode before the positioning > operation (note that for implementations allowing my first model of > append, the one Randy thinks is most natural, and the one I think that > the RM specifies now, append mode is equivalent in any case to just > opening in write mode and positioning to the end manually). True enough. Append_File mode should never have been added to stream files in the first place: it adds nothing, and it is dangerous to boot (if people think that they can do POSIX O_APPEND with it, which almost never will work). **************************************************************** From: Robert Dewar Sent: Thursday, January 18, 2001 8:52 PM Randy you missed my point completely I am afraid, I was wondering how many applications depend on O_APPEND semantics, sorry if I was not clear. The critical thing about O_APPEND is the safety. **************************************************************** From: Randy Brukardt Sent: Thursday, January 18, 2001 10:05 PM Humm, perhaps you missed my point, which is that for a Stream_IO file, there almost never *is* any additional safety from using O_APPEND. That's because a typical stream write is made up of many tiny writes, and another process writing the same file would still jumble it up. It would only work if no other process did write the file; but then you don't need O_APPEND semantics! Sequential files don't have this problem; it really only occurs with stream files. (For a text log, it possibly could happen depending on how Text_IO is designed, but it's unlikely to be fatal.) The only time O_APPEND could be useful would be when the file consists *only* of elementary types *and* the file needs to be positioned (presumably in another mode). This seems unlikely enough to be the empty set. (Log files are typically text files, and wouldn't be written with Stream_IO in the first place.) **************************************************************** From: Pascal Leroy Sent: Friday, January 19, 2001 2:32 AM Your reasoning is correct if you use T'Write or T'Output to write to a Stream_IO file. But on the other hand, it is perfectly possible to write to a Stream_IO file using only Stream_IO.Write (perhaps because you do conversion to byte arrays yourself), in which case the atomicity property of O_APPEND is applicable. **************************************************************** From: Robert Dewar [dewar@gnat.com] Sent: Friday, January 19, 2001 6:36 AM Obviously you do NOT use "many tiny writes" if you are using Stream_IO with O_APPEND and want indivisible semantics. You use one write of a stream element array of appropriate length. **************************************************************** From: Robert Dewar Sent: Friday, January 19, 2001 6:39 AM <> This seems a REALLY strange claim. My version of the RM has: procedure Write (File : in File_Type; Item : in Stream_Element_Array; To : in Positive_Count); And allows arbitrarily long Item's to be written. yes, of course you have to avoid doing individual little writes to the Stream of the file if you want O_APPEND semantics for logging. But that's easy to program. Now of course if people naively DO use the Stream, yes, then the whole O_APPEND argument is junk. Furthermore, if you are going to use the Write above, you could in practice equally well use O_APPEND mode for sequential IO instantiated for Stream_Element_Array (although that would put extra "green word" junk into the file, which might be unacceptable). Still, I agree, this makes it much less likely that there really IS anyone depending on O_APPEND semantics out there after all. So I guess the argument for switching to allowing O_APPEND semantics is much weaker. **************************************************************** From: Ted Baker Sent: Friday, January 19, 2001 6:33 AM | ... it is very likely that the two records would end up interspersed, and | they could not be read by any means at all ... You seem to have missed what is atomic here. It is not the writing of a block of data as an atomic block. What is atomic is the positioning to the end being atomic with the writing of whatever is the (implementation-dependent) atomically writable unit of data. What you gain is that you will NEVER OVERWRITE anything, because the positioning is atomic with the writing. Thus, no data will be lost, even if some other thread of control tries to change the file position. | ... atomicity of O_APPEND is not useful for Stream_IO | files ... OK. You are correct that there may be interleaving, and that argues well against the utility of this feature with Ada streams. The places where the append semantics are most useful are where data is being written in chunks (a buffer full), and the maximum atomically writeable chunk will be implementation defined (usually at least 512 bytes on disk files). **************************************************************** From: Robert Dewar Sent: Friday, January 19, 2001 6:45 AM <> Right, same point I just made (sorry for duplication), but you have got to admit that in practice it is pretty unlikely that anyone is actually going to this approach. **************************************************************** From: Ted Baker Sent: Saturday, January 20, 2001 10:30 AM | Robert Dewar wrote: | > ... I guess at this stage, if we really want to | > allow the O_APPEND implementations to be considered conforming, we just | > bite the bullet and say that it is implementation defined whether | > positioning is allowed for stream files in append mode -- seems pretty | > ugly, but I see no alternative. | We should be careful here to allow what to me is the most natural | expectation, namely that positioning is allowed, but the position is | relative to the part you are adding. That is really strange! What is "the part you are appending"???? Logically, each write operation to the file appends a new "part". Since the standard is effectively silent on the definition of "append", but uses the term to say that in Append_File mode "only appending" is to be peformed, it is very clear that the dictionary definition must apply. That means each write operation to the file must add on the output to the end of the file: no overwriting or truncation is possible. To accomplish this you have two choices: 1) Allow the reader to do "positioning", as POSIX does, but require that this repositioning have no effect on where the output goes; the data still gets written at the end of file. 2) Forbid repositioning on files in Append_File mode. Both of these allow (but to not require) the implementation to use O_APPEND mode. **************************************************************** From: dewar@GNAT.COM Sent: Saturday, January 20, 2001 10:41 AM <> This is quite misleading, Ted, have you read the whole thread here. TO argue that the RM is clear and unambiguous at this point is very dubious. In particular, Append_File mode IS defined at several places to mean that the effect is to position to the end of the file, and there appears to be a quite deliberate exclusion of stream_io (the only case nder discussion -- please remember this) from the statement that all writes are to the end of the file. Please read again my description of why my model of appending is perfectly reasonable according to the dictionary definition, (it's the analogy of writing things by hand in response to a suggestion that you append a note to the end of what's written -- there is no requirement that your appended note be written in a compeltely sequential manner. Furthermore, we have this VERY important point that in the normal mode of operation of Stream_IO, namely using the stream attributes, the whole idea of process-safe appending is broken in any case. **************************************************************** From: Ted Baker Sent: Saturday, January 20, 2001 11:41 AM | This is quite misleading, Ted, have you read the whole thread here. I've gone back and I think I've read all of it, though I have read it pretty much in reverse order, due an intervening busy period here. | In particular, Append_File mode IS defined at several places to mean | that the effect is to position to the end of the file, and there appears I used emacs to search through the RM for such places, before my last e-mail. Not much is said: Append_File: | ... These values correspond respectively to the cases where only | reading, only writing, or only appending are to be performed. ============== This seems to be the most definitive statement of the semantics of Append_File. Open: | ... When the file is opened with mode Append_File, transfer to the file | starts after the last element of the file. This simply confirms what append means. Close: | ... for sequential files, if the file being closed has mode Out_File or | Append_File, then the last element written since the most recent open | or reset is the last element that can be read from the file. This confirms that if one somehow were able to overwrite a portion of the file in Append_File mode, and that were the last operation done, the rest of the file must be truncated. To me, this further underscores the requirement that in that mode all output must be to the end of the file, i.e., either there is no repositioning, or positioning has no effect. Reset: | ... writing to its elements can be restarted ... after the last | element of the file (for mode Append_File). ... In addition, for | sequential files, if the given file has mode Out_File or | Append_File when Reset is called, the last element written since | the most recent open or reset is the last element that can be read | from the file. Se the comment above on Close. Write: | 6 Operates on a file of mode Out_File or Append_File. Writes the | value of Item to the given file. Adds nothing special for Append_File. | 17 (20) Append_File mode is not supported for the generic package | Direct_IO. Further supports the idea that Append_File cannot be used in conjunction with random-access/respositioning. [skipping stuff that relates only to text I/O, page and line lengths, etc.] | Set_Mode: | ... If the new mode is Append_File, the file is positioned to its | end ... Consistent with the requirement that all operations can only append to the file. | to be a quite deliberate exclusion of stream_io (the only case nder | discussion -- please remember this) from the statement that all writes | are to the end of the file. Where do you see this as an exclusion? | 10 type File_Mode is (In_File, Out_File, Append_File); | -- for Sequential_IO, Text_IO, Wide_Text_IO, and Stream_IO | 11 These values correspond respectively to the cases where only | reading, only writing, or only appending are to be performed. ============== | 12 The mode of a file can be changed. | Please read again my description of why my model of appending is perfectly | reasonable according to the dictionary definition, (it's the analogy | of writing things by hand in response to a suggestion that you append | a note to the end of what's written -- there is no requirement that | your appended note be written in a compeltely sequential manner. That is a big stretch, especially given the explicit statements about the truncation that happens when you close or reset the file (see above) if the last element written is not at the end of the file. That requires at least some restriction on order of operations. | Furthermore, we have this VERY important point that in the normal | mode of operation of Stream_IO, namely using the stream attributes, | the whole idea of process-safe appending is broken in any case. Granted you can't get process-safe appending in that case, but I believe you still ought to allow (not requir) an implementation to use O_APPEND to implement this mode. The way to do that is to either completely disallow positioning for files in Append_File mode, or to specify that such positioning has no effect on output of such files-- the output still is appended (to the end). **************************************************************** From: dewar@GNAT.COM Sent: Saturday, January 20, 2001 11:53 AM <> No, in RM terms, it *defines* what Append mode means. Since this is the only defined effect, this is the ONLY effect. That's the way the RM works. <<| ... for sequential files, if the file being closed has mode Out_File or | Append_File, then the last element written since the most recent open | or reset is the last element that can be read from the file. This confirms that if one somehow were able to overwrite a portion of the file in Append_File mode, and that were the last operation done, the rest of the file must be truncated. >> This really rates a huh? :-) This is a statement about *SEQUENTIAL FILES* we are not talking about sequential files! We are talking about stream files. Indeed it is the specific exclusion of stream files that is significant here. If the authors of the RM had intended your interpretation, they would have omitted the qualifer "for sequential files". Thus this last quote, which you seem to think supports your position, in fact is the strongest evidence internal to the RM that supports the position you disagree with! Ted, please focus on the fact that this entire discussion is about stream files. That's an important point here, because we all agree that sequential files COULD be handled with O_APPEND, though of course there is no such requirement, and any program that depends on this is relying on undocumented implementation dependent (not even implementation defined) behavior. **************************************************************** From: dewar@GNAT.COM Sent: Saturday, January 20, 2001 10:57 AM <<1) Allow the reader to do "positioning", as POSIX does, but require that this repositioning have no effect on where the output goes; the data still gets written at the end of file.>> There is no conceivable justification for this interpretation from the definitions in the RM. **************************************************************** From: Ada Rapporteur Group of WG9; managing the Ada issues [ARG@ACM.ORG] on behalf of dewar@GNAT.COM Sent: Saturday, January 20, 2001 12:45 PM To: ARG@ACM.ORG Subject: Re: AI-85 analysis and discussion. <<11 These values correspond respectively to the cases where only reading, only writing, or only appending are to be performed. >> Yes, but that says nothing, it does not define what appending means, only that this is what happens after such an open. So we have to lool elsewhere in the RM for what it means. And we find that for stream files, it means that an initial open positions you at the end, as does a reset, and for sequential files, that writes go at the end. But there is NO statement that writes for stream files go at the end. Your only argument for this is that you think it should be this way, but there is nothing in the RM to support this view. Now we can speculate, as Pascal has interestingly done, on historical reasons why the RM is the way it is, and perhaps that it does not map the original intent, which is always a fair discussin. But your argument that the RM clearly requires stream file writes to be done at the end definitely does NOT hold water. << 28 The subprograms Create, Open, Close, Delete, Reset, Mode, Name, Form, =========== Is_Open, and End_of_File have the same effect as the corresponding =============== subprograms in Sequential_IO (see A.8.2). >> Yes, indeed, but unfortuantely we are talking about the effect of write, and positioning operations, which are NOT included in the above list. Yes, Open *DOES* have the same effect. As for close, the definition of close is: 10 Severs the association between the given file and its associated external file. The given file is left closed. In addition, for sequential files, if the file being closed has mode Out_File or Append_File, then the last element written since the most recent open or reset is the last element that can be read from the file. If no elements have been written and the file mode is Out_File, then the closed file is empty. If no elements have been written and the file mode is Append_File, then the closed file is unchanged. And we see very clearly that the rule about last element written being the last element applies only for sequential files. It could not be much clearer! Yes, it is a bit odd to say that the behavior is the same for A and B, when the rule you are citing specifically says that A and B behave differently, and in Pascal's mode, you could site this as additional evidence that the phrase "for sequential files" here was from the time when stream files were considered to be sequential files, and someone forgot to fix it. Also, how can you explain, if your meaning is so self-evident, that there is no provision for preventing positioning operations in append mode, if they are completely meaningless, as you contend. I am not saying that your desired interpretation is unreasonable, it is indeed one of the positions we are considering, but to argue that the RM unambguously specifies this behavior (and thus presumably requires no modification in this area) is just not supported by the quoted text. It is also not clear to what extent you are arguing with O_APPEND in mind. Obviously the RM does not require O_APPEND behavior in any case, and indeed, as we have discussed, O_APPEND and stream_io are not very compatible, in that the normal mode for stream io is lots o little writes which would break the guarantee of indivisibility in any case. Given that O_APPEND is not *required*, the question is whether to *allow* it at all. There are several possible answers: 1. No O_APPEND cannot be used, because positioning must be possible. 2. (more reasonable). Of coruse O_APPEND can be used, but if someone does a positioning operation, then you have to implicitly reopen in update mode. By the way, in GNAT we do this kind of reopening anyway on a reset, because you don't want to have to open in update mode just in case you do writes. 3. No positioing is allowed, so O_APPEND is OK, this has two subcases 3a. We make it clear that a given implementation is allowed to restrict positioning on any files it cares to. Two subcases 3a1. On all files (Tuck's view, I disagree) 3a2. On stream files in append mode 3b. We rewrite the RM to prohibit positioning on stream files in append mode (or if you take Ted's view, no rewriting is needed because he thinks that's what the RM says now). 4. (Least convincing, though perhaps this is what Ted thinks the RM says now). Positioning on stream files is allowed but does not affect subsequent write operations. We need to a) choose one of these positions b) figure out what the RM should be saying to support this position I really don't think it is very helpful to argue that both the ltter and intent of the RM clearly specify *ANY* of these positions, because if they did, we would not be having this discussion. **************************************************************** From: Ted Baker Sent: Saturday, January 20, 2001 12:23 PM I see you chose to reply only to the parts of my message where you could pick fault. I guess that means you agreed with the rest. | <> | No, in RM terms, it *defines* what Append mode means. Since this is the | only defined effect, this is the ONLY effect. That's the way the RM | works. No. It only defines part of what Append mode means. We also have: -- for Sequential_IO, Text_IO, Wide_Text_IO, and Stream_IO ========= 11 These values correspond respectively to the cases where only reading, only writing, or only appending are to be performed. ============== | <<| ... for sequential files, if the file being closed has mode Out_File or | | Append_File, then the last element written since the most recent open | | or reset is the last element that can be read from the file. | This confirms that if one somehow were able to overwrite | a portion of the file in Append_File mode, and that were the | last operation done, the rest of the file must be truncated. | >> | This really rates a huh? :-) | This is a statement about *SEQUENTIAL FILES* ... Huh? :-) Read your RM again, the section on Stream_IO: 28 The subprograms Create, Open, Close, Delete, Reset, Mode, Name, Form, =========== Is_Open, and End_of_File have the same effect as the corresponding =============== subprograms in Sequential_IO (see A.8.2). ============= | Ted, please focus on the fact that this entire discussion is about stream | files. That's an important point here, because we all agree that sequential | files COULD be handled with O_APPEND ... Of course. However, I think we all agree that the RM did not do a good job of explicitly stating the semantics of Append_File with stream files. Therefore, I believe it is appropriate to appeal to analogies with other uses of Append_File. The RM text on stream files itself refers back to the concepts of sequential and direct files. In addition to the cited text above, which refers back to Sequential_IO for the effects of Open and Close, we have: "Access to a stream file is either sequential, via a call on Read or Write to transfer an array of stream elements, or positional (if supported by the implementation for the given file), by specifying a relative index for an element." It seems clear to me that mode Append_File only makes sense when the access is sequential, since it is not allowed for direct files. **************************************************************** From: dewar@GNAT.COM Sent: Saturday, January 20, 2001 12:47 PM <> you guess wrong! I was responding only to the specfic quotes from the RM, because that's the only relevant issue at this stage if we are talking about what the RM means. The fact that Ted Baker thinks it should mean something else or that Robert or Pascal or anyone else does, is not really relevant to the issue of trying to clean this up. **************************************************************** From: Ted Baker Sent: Saturday, January 20, 2001 1:49 PM | Yes, it is a bit odd to say that the behavior is the same for A and B, | when the rule you are citing specifically says that A and B behave | differently, and in Pascal's mode, you could site this as additional | evidence that the phrase "for sequential files" here was from the | time when stream files were considered to be sequential files, and | someone forgot to fix it. Yes, I suppose Pascal and I are arguing in the same direction here, i.e., that spelling out the semantics of Append_File mode for stream files was unintentionally neglected when stream files were extended to allow non-sequential access. What we have now for stream files is a union of sequential and direct file operations. My contention is that where there is a gap in the description of stream files the semantics for sequential files should be applied to the sequential io operations, and the semantics for direct files should be applied to the direct io opeations. | I am not saying that your desired interpretation is unreasonable, it is | indeed one of the positions we are considering, but to argue that the | RM unambguously specifies this behavior (and thus presumably requires | no modification in this area) is just not supported by the quoted text. I'm willing to concede that I overstated the support in the ARM for this view, and that some clarification is needed. | Given that O_APPEND is not *required*, the question is whether to *allow* | it at all. There are several possible answers: | 1. No O_APPEND cannot be used, because positioning must be possible. I've been arguing against the above. | 2. (more reasonable). Of coruse O_APPEND can be used, but if someone | does a positioning operation, then you have to implicitly reopen in | update mode. By the way, in GNAT we do this kind of reopening anyway | on a reset, because you don't want to have to open in update mode | just in case you do writes. I would not object to this strongly, but it seems to violate the rule of least surprise, i.e., by making repositioning act like a reopen. | 3. No positioing is allowed, so O_APPEND is OK, this has two subcases | 3a. We make it clear that a given implementation is allowed to | restrict positioning on any files it cares to. Two subcases | 3a1. On all files (Tuck's view, I disagree) I guess you don't disagree that we can't reposition a UNIX terminal device (character special file). | 3a2. On stream files in append mode Seems reasonable to me. | 3b. We rewrite the RM to prohibit positioning on stream files in | append mode (or if you take Ted's view, no rewriting is needed | because he thinks that's what the RM says now). This is what I would prefer, because I think that is most consistent with the rest of what is in the RM. | 4. (Least convincing, though perhaps this is what Ted thinks the RM | says now). Positioning on stream files is allowed but does not | affect subsequent write operations. I think this would also be OK, but is less consistent with what the RM says. | We need to | a) choose one of these positions | b) figure out what the RM should be saying to support this position | I really don't think it is very helpful to argue that both the ltter | and intent of the RM clearly specify *ANY* of these positions, because | if they did, we would not be having this discussion. OK. It is not clear enough, and needs to be specified. **************************************************************** From: dewar@GNAT.COM Sent: Saturday, January 20, 2001 2:12 PM <> No, I agree with that (and indeed interpreted the phrase about files not allowing positioning to be *exclusively* talking about this case :-) <> The argument in favor is that it is upwards compatible for implementations that do allow positioning (and I think we have to agree that the current RM wording does *allow* positioning oeprations on stream files in append mode since it does not disallow it). A nasty mess. I must say I really VERY much doubt that anyone is using the native write operations in append mode on an O_APPEND implementation today to ensure process serialization of output. It is just *so* convenient to use the stream attributes, and that's what one does 99.9% of the time. It is a real pain to do otherwise (you have to instantiate your own implementation of streams that generates buffers that can then be setup using stream attributes, and then manually write these buffers with direct calls to Stream_IO.Write -- yes it can be done, but I would be surprised if it is done. Indeed it seems rather error prone to me to even suggest that O_APPEND mode will be used for append mode with streams, since it is SO easily misused. Interestingly, in GNAT we don't use O_APPEND for anything (though we do have a general mechanism for doing Ada I/O operations on files that are attached to streams opened with C calls, so someone can do this for them selves). We don't even use O_APPEND for sequential files (there is no requirement to do so), and we never got a suggestion to change this, let alone a problem report. ****************************************************************