Version 1.1 of ai12s/ai12-0340-1.txt

From: Tucker Taft
Sent: Thursday, June 20, 2019  12:22 PM

>> ...
>> Might be clearer 
>> if you called the routine "Put_Image" rather than "Image".  
> 
> Agreed. Why confuse the usage (usually a routine "Image" returns a String).
[Editor's note: The quoted text comes from a thread associated with AI12-0312-1.]

By the way, we have had an internal comment that Put_Image as specified takes
a stream, but in fact it probably fails miserably if you write anything but 
Wide_Wide_Character and Wide_Wide_String on the stream.  That admittedly seems
pretty error-prone.

The suggestion is that we define a new kind of "Text_Stream," probably in 
Text_IO.Text_Streams, which allows you to write/read any size of character, or
any string thereof, and it does the "right" thing, as far as storing the 
characters as Wide_Wide_Characters, or more efficiently, using UTF8.  'Image 
would be defined using a Storage_Text_Stream presumably.  There would probably
also be a New_Line procedure and an End_Of_Line function so that you don't 
have to write Wide_Wide_Character'(ASCII.LF) or equivalent to create a 
multi-line image.

A Text_Stream would presumably be an extension of Root_Stream_Type, and 
function stream of *Text_IO.Text_Streams might actually return an access
Text_Stream'Class instead of what it is returning now.

Clearly needs some work, but something like this could reduce the errors 
inherent in the current model, where only Wide_Wide_Character and 
Wide_Wide_String actually work correctly with a Put_Image stream.  Brad, for 
example, seems to tripped over this in his example.

****************************************************************

From: Randy Brukardt
Sent: Thursday, June 20, 2019  11:26 PM

Two thoughts on this one (not well thought out yet).

(1) The reason that we switched to the streaming model was to avoid overhead. 
Adding an additional level of dispatching calls clearly adds overhead (and a 
lot of it if UTF-8 is used internally).

(2) This description seems to be confusing a stream (which is an I/O or 
buffering concept on raw bytes with a very simple interface), and the 
somewhat higher level features that read/write a stream or buffer (the stream
attributes).

If this is a serious concern (which is not obvious to me, few people ought to 
need to write Put_Images), then clearly streaming is the wrong model for 
Image. With this concept, there can't be any streaming involved in order that 
the overhead remain manageable (a multi-level way would be crazy). That makes 
this suggestion something different (essentially, a fancy text buffer), and if
it goes forward it should be named and described differently (calling it a 
stream would just be confusing). It might make sense to allow this buffer to
(optionally) be read/written to a stream, but that wouldn't be the primary 
purpose or use. (That also would solve the Global problem for it and 'Image, 
as a text buffer does not need to read/write any globals.)

Sounds like a substantial job, though, and a complete redo of the Image stuff 
(which was very painful the first time). And we'd lose the main justification 
for the Fifo_Streams -- should they even be kept around??

****************************************************************

From: Randy Brukardt
Sent: Friday, June 21, 2019  1:32 AM

BTW, Ada already has something called Text_Streams - see A.10.1 . And that's 
more in line with streaming than this proposal.

****************************************************************

From: Brad Moore
Sent: Saturday, June 22, 2019  1:45 AM

I have some thoughts on this issue as well. I like Tucker's suggestion of a
special kind of queue

First some observations. The Streams mechanism appears to be much broader 
machinery than what is needed. We really need only a small subset of 
Ada.Streams capabilities.

1) The programmer only need a stream that supports writes. The implementation 
hides the reading of the stream, and provides the actual stream object and 
class. The implementation doesn't even need to be use the streaming mechanism 
to pull the data out of the stream. At that point it is effectively just an 
array of wide-wide characters, which could be extracted directly.

2) The programmer only needs a stream that supports writes of one type, 
wide-wide-characters. The streaming mechanism allows writes of most any type.

It seems what we really want is a queue that mostly acts like a stream, but 
where all objects in the queue are of the same type (homogeneous).

I have written a composable set of queues/deques generics called dequesterity 
which can be found on SourceForge, that support many different types of queues
including streaming queues, and just regular queues. They all have mostly a 
common interface, but only the streamable queues allow me to use the 'Read and
'Write attributes, which seems like an unfortunate limitation. 

It would be nice if 'Read and 'Write attributes could be generalized to support 
other types of queues (homogeneous ones).

Here are some ideas floating around in my head.....

Ada.Containers.Synchronized_Queue_Interfaces seems much closer to what we 
need, than Ada.Streams.

Perhaps we could use that, or if we don't want synchronization, define a 
similar Ada.Containers.Unsynchronized_Queue_Interfaces package?

What if we could apply 'Read and 'Write to such containers based on such 
interfaces to read and write objects of Element_Type?

We could create a standard instance using Wide_Wide_Character, e.g.

package Ada.Containers.Wide_Wide_Character_Queues is
  new Ada.Containers.Synchronized_Queue_Interfaces
    (Element_Type => Wide_Wide_Character);

And then use an object of;

   not null access Ada.Containers.Wide_Wide_Character_Queue.Queue'Class 

as the parameter of the call associated with 'Put_Image.

'Read would map to a call to Dequeue, and 'Write would map to a call to 
Enqueue.

This way, the compiler could ensure statically that the only 
Wide_Wide_Characters are being written for Put_Image.
We'd get to use 'Read and 'Write more generally, and could potentially extend 
to be applied to user-defined homogeneous queues.

Its also nice that a programmer could use the same subprograms they'd write 
for this specialized streaming function to also work with regular Ada.Streams
objects.

Anyway, there may be giant holes in this idea, and obviously needs to be 
fleshed out further but I think it could fit in with Tucker and Randy's 
thoughts, and it at least seems workable to me....

****************************************************************

From: Tucker Taft
Sent: Saturday, June 22, 2019  1:28 PM

>BTW, Ada already has something called Text_Streams - see A.10.1.

Yes, I realized that, and was suggesting this new kind of stream might be 
added to that existing package.

>And that's more in line with streaming than this proposal.

It is solving a different problem, namely creating a stream from an existing 
open Text_IO.File.  This new "text" stream would have an interface that had 
some additional operations, some of which might be class-wide rather than 
dispatching, conceivably, to unify normal, wide, and wide-wide character 
streaming into something like UTF8, or some other universal format (UTF16,
UCS32).  In other languages, there are various "wrappers" you can apply to
a stream to create a buffered stream, a formatted stream, etc.  Clearly this
is a bigger topic than just Put_Image, but Put_Image seems to have brought it
to the fore.

****************************************************************

From: Randy Brukardt
Sent: Monday, June 24, 2019  5:45 PM

...
> It would be nice if 'Read and 'Write attributes could be generalized 
> to support other types of queues (homogeneous ones).

I don't see how that could work. These attributes take a stream parameter and 
use dispatching operations of a stream to do their work. You seem to be 
suggesting that they also somehow take a queue parameter instead and 
presumably use dispatching operations of a queue to do their work.

That could only happen if the entire thing was duplicated (overloaded). I 
suppose a compiler could do that, but it seems like a lot of overhead. And I 
have no idea how a user-defined 'Read or 'Write would work in such a case - it
would seem that two routines would be needed.

If you somehow tried to use the stream interface to read/write to the queues, 
you'd lose any type checking, which would seem to defeat the purpose.

****************************************************************

From: Randy Brukardt
Sent: Monday, June 24, 2019  6:47 PM

...
>>. And that's more in line with streaming than this proposal.
>
>It is solving a different problem, namely creating a stream from an 
>existing open Text_IO.File.  This new "text" stream would have an 
>interface that had some additional operations, some of which might be 
>class-wide rather than dispatching, conceivably, to unify normal, wide, 
>and wide-wide character streaming into something like UTF8, or some 
>other universal format (UTF16, UCS32).

This is indeed what I thought you were talking about, and while I have little
objection with the concept, it seems wrong to call it a "stream" (or to use 
Ada streams) with such a construct. This objection stems from the way Ada 
currently uses these terms, but also in that you are making something that
can and is safe into something unsafe and much more error-prone.

>In other languages, there are various "wrappers" you can apply to a 
>stream to create a buffered stream, a formatted stream, etc.  Clearly 
>this is a bigger topic than just Put_Image, but Put_Image seems to have 
>brought it to the fore.

Given the design of Ada Streams, this is bass-ackwards. Ada Streams is a 
completely untyped, you're on your own mechanism. It is hard to avoid causing 
erroneous execution using Ada Streams. One cannot build safe abstractions on 
top of such a basis unless you can totally hide the underlying mechanism. But
Ada doesn't let one hide underlying abstractions; they're pretty much only 
additive. (Raising an exception in this case wouldn't work, either, as one 
would need the underlying abstraction to implement the guy.)

And there is absolutely no reason for a text buffer to have the possibility of 
erroneous execution. If one sticks a bunch of strings into a buffer, the result
can be read as one or more strings. (I'm ignoring any meaning, of course - that
an't make a program become erroneous.)

There's also the obvious possibility of confusion. If you have the Write 
attribute and a Write subprogram, it's easy to imagine getting them mixed up. 
(Choosing the names better could help a bit, but not a lot -- people are used 
to using 'Write to write a stream, but here that would be wrong.) And if the 
buffer is converting to UTF-8 internally, they are going to do something 
pretty different for every built-in Ada type.

I would much prefer to see a strongly typed interface that doesn't allow 
anything other that strings into and out of the buffer, especially if you are 
doing conversions. Why encourage misuse?? I could see allowing the buffer to 
be streamed into or out of (which would be untyped, of course), but not 
treating it as an Ada Stream.

...

>>then clearly streaming is the wrong model for Image.

>I think a stream is OK, but the question is how is the 
>stream interpreted.   

If you are interpreting a stream as anything other than a raw stream of bytes, 
you are making matters worse, not better.

>I would have presumed that a typical use of Put_Image would be to give 
>it a stream from, say, Text_IO.Text_Streams.Stream, and then directly 
>put the image onto the stream rather than doing a Put(File, 
>Blah'Image).  But that probably won't work.
>In fact, I think as defined you would have to use 
>Wide_Wide_Text_IO.Text_Streams.Stream to produce a stream that would 
>work with Put_Image.

I can't imagine why you would mess with streams for this, even if you had to 
use Text_IO (and that's not that likely). Dragging in Text_IO is a massive 
pile-o-stuff that one doesn't want to require in Ada programs.

>I am thinking that perhaps Wide_Wide_Text_IO.Text_Streams, 
>Wide_Text_IO.Text_Streams, and Text_IO.Text_Streams should all produce 
>a stream that is "universal" in some sense, and can accept Characters, 
>Wide_Characters, and Wide_Wide_Characters
>and do the "right" thing.   

That would be wildly incompatible, given the existing byte stream 
intepretation of these things.

Yes, there's an argument that you should have done all of this in Ada 95, but 
you would have had to have known about UTF-8 and how it would end up 
dominating back then to understand what to do. And had a lot more will about 
abandoning type String back then, too. It's too late now, unless you want to 
make a parallel runtime.

...
>>Sounds like a substantial job, though, and a complete redo of the 
>>Image stuff (which was very painful the first time). And we'd lose the 
>>main justification for the Fifo_Streams -- should they even be kept around??
			
>I think Fifo_Streams are part of the solution...

No, because they're untyped. Something very much like them could be the 
solution, but it would have to be typed (text-only).

>... , but we agree that this will need more thought.  I don't see this 
>as a complete "redo" of the image stuff.  All I am hoping is that we 
>have the ability to use normal Strings/Characters rather than only 
>Wide_Wide_Strings/Characters to produce an image.

"Complete" was probably too strong. It just means completely removing all of 
the stream stuff from Image and using a typed text buffer instead. I have 
come to think this would be a good thing, as it would solve the 
Nonblocking/Global issues as well (no streams, no dispatching nonsense).

****************************************************************

From: Randy Brukardt
Sent: Monday, June 24, 2019  6:56 PM

> This is indeed what I thought you were talking about, and while I have 
> little objection with the concept, it seems wrong to call it a 
> "stream" (or to use Ada streams) with such a construct. This objection 
> stems from the way Ada currently uses these terms, but also in that 
> you are making something that can and is safe into something unsafe 
> and much more error-prone.

In the interests of something positive, let me propose a specification for 
a "Text_Buffer" that does something like Tucker describes. It would allow 
concatenating chunks of text in whatever format as well as extracting them
in similar arbitray formats. It could have an optional streaming interface,
but it could be used without referring to any streams (and usually would be
used that way).

Presumably, there would also be a bounded version (not shown).

Something like the following would be ideal:

    package Text_Buffers
       with Preelaborated, Nonblocking, Global => null is
           -- This Global => null requires that a Text_Buffer is a compound object,
           -- and any access types be internal.

         type Text_Buffer is private;

         function Character_Count (Buffer : in Text_Buffer) return Natural;

         procedure Clear (Buffer : in out Text_Buffer);

         procedure Write (Buffer : in out Text_Buffer;
                          Item   : in String)
            with Post => Character_Count (Buffer)'Old + Item'Length = 
                         Character_Count (Buffer);

         procedure Wide_Write (Buffer : in out Text_Buffer;
                               Item   : in Wide_String)
            with Post => Character_Count (Buffer)'Old + Item'Length = 
                         Character_Count (Buffer);
            -- I hate having to use "Wide_xxx" here, but it's necessary if we're
            -- going to allow using literals conveniently with this package.

         procedure Wide_Wide_Write (Buffer : in out Text_Buffer;
                                    Item   : in Wide_Wide_String)
            with Post => Character_Count (Buffer)'Old + Item'Length = 
                         Character_Count (Buffer);

         procedure Read (Buffer : in out Text_Buffer;
                         Item   : in out String;
                         Length : in out Natural)
            with Post => Character_Count (Buffer)'Old - Length = 
                         Character_Count (Buffer);

         function Read (Buffer : in out Text_Buffer) return String
            with Post => Character_Count (Buffer)'Old = Read'Length and then
                         Character_Count (Buffer) = 0;

         -- Same for Wide_String and Wide_Wide_String.

         -- We also could have UTF8 and UTF16 routines (possibly in a child,

         -- to avoid dragging in Ada.Strings.Encoding unless needed).

         -- Similarly, we could support a streaming interface:

         procedure Read_into_Buffer(
            Stream : not null access Ada.Streams.Root_Stream_Type'Class;
            Buffer : in out Text_Buffer;
            Length : in Natural)
            with Post => Character_Count (Buffer)'Old + Length = 
                         Character_Count (Buffer)
                 Nonblocking => False, Global => in out all;       
            -- Read length Wide_Wide_Characters from Stream.

         procedure Write_from_Buffer(
            Stream : not null access Ada.Streams.Root_Stream_Type'Class;
            Buffer : in out Text_Buffer;
            Length : in Natural := 0)
            with Post => (if Length /= 0 then
                             Character_Count (Buffer)'Old - Length = 
                             Character_Count (Buffer)
                          else  
                             Character_Count (Buffer) = 0)     
                 Nonblocking => False, Global => in out all;       
            -- Write length Wide_Wide_Characters to Stream. If Length is 0, 
		-- write all of the characters in the buffer to the stream.

================

Put_Image would then be modified to take a Text_Buffer rather than a Stream:

procedure S'Put_Image
   (Buffer : in out Text_Buffer'Class;
    Arg    : in T);


Thoughts and brickbats welcome.

****************************************************************

From: Tucker Taft
Sent: Monday, June 24, 2019  8:46 PM

Interesting.  It looks like you are assuming that Text_Buffer is tagged.  You 
could imagine an implementation of Text_Buffer on top of a "standard" stream, 
but you make the good point that you don't want both a "standard" stream and 
a "formatted" stream interface available on the same object.  The original 
comment recommended providing a "New_Line" operation to allow the 'Image to
have line breaks in it.  Implies you might want an End_Of_Line query as well.

So do folks feel we should move in this direction for Put_Image?

****************************************************************

From: Randy Brukardt
Sent: Monday, June 24, 2019  9:37 PM

> Interesting.  It looks like you are assuming that Text_Buffer is 
> tagged.

I thought I had written "tagged private", but I must have taken that out at 
some point. And then depended upon it. :-(

I was presuming that we still would want a bounded version, and clearly 
Put_Image has to work with both. So that implies dispatching calls, and thus 
tags.

> You could imagine an implementation of Text_Buffer on top of a 
> "standard" stream, but you make the good point that you don't want 
> both a "standard" stream and a "formatted" stream interface available 
> on the same object.
> The original comment recommended providing a "New_Line" 
> operation to allow the 'Image to have line breaks in it.  
> Implies you might want an End_Of_Line query as well.

Suppose that makes sense.

> So do folks feel we should move in this direction for Put_Image?

Seems to help in usage (Brad demonstrated this is easy to get wrong) and as 
a side-effect reduces the issues of blocking, side-effect-full 'Image.

****************************************************************

From: Brad Moore
Sent: Tuesday, June 25, 2019  8:35 PM

> ...
>> It would be nice if 'Read and 'Write attributes could be generalized 
>> to support other types of queues (homogeneous ones).
> 
> I don't see how that could work. These attributes take a stream 
> parameter and use dispatching operations of a stream to do their work. 
> You seem to be suggesting that they also somehow take a queue 
> parameter instead and presumably use dispatching operations of a queue to do
> their work.
> 
> That could only happen if the entire thing was duplicated 
> (overloaded). I suppose a compiler could do that, but it seems like a 
> lot of overhead. And I have no idea how a user-defined 'Read or 'Write 
> would work in such a case - it would seem that two routines would be needed.

Yes, I was thinking of overloading the attributes such that if the argument is 
a homogeneous queue type, it would get mapped to the Enqueue/Dequeue calls, 
without invoking any of the streaming machinery. 

Streams are effectively heterogenous queue abstractions (they can put objects 
of different types in the queue). I was suggesting using some syntactic sugar 
to extend 'Read and 'Write to also work with homogeneous queue abstractions.
To the user, the semantics of a queue would be similar even though the 
underlying implementation would be very different.

In any case, the syntactic sugar of using 'Read and 'Write isn't critical. If 
we didn't have that, one could just call the Enqueue/Dequeue primitives 
directly, which I think falls in line more or less with what you are 
proposing, which is to have a special interface abstraction for this, rather 
than use the streaming mechanism.

> If you somehow tried to use the stream interface to read/write to the 
> queues, you'd lose any type checking, which would seem to defeat the 
> purpose.

I was thinking that the compiler would enforce that only one type could be 
streamed into such a queue.

The overloading of 'Read and 'Write was just a side thought, to see if it 
might generate any interest.
So far it seems like it does not hold any appeal, which is Ok. What's 
important is to have a mechanism that isn't error prone that doesn't feels 
like sticking a round peg into a square hole.

****************************************************************

From: Brad Moore
Sent: Monday, June 24, 2019  8:37 PM

> So do folks feel we should move in this direction for Put_Image?

I also agree that this seems like a step in the right direction.

****************************************************************