Version 1.1 of acs/ac-00305.txt

Unformatted version of acs/ac-00305.txt version 1.1
Other versions for file acs/ac-00305.txt

!standard 13.2          18-02-21 AC95-00305/00
!standard 13.5.1
!class Amendment 18-02-21
!status received no action 18-02-21
!status received 18-01-15
!subject Bit Order issues
!summary
!appendix

!topic Composable records with non-default bit order
!reference Ada 2012 RM13.5.1
!from Niklas Holsti 18-01-15
!keywords record representation bit-order
!discussion

This is either a question regarding the semantics of record
representation clauses with the non-default Bit_Order aspect, or a
suggestion for an improvement to those semantics. I admit that I do not
fully understand the current definition, but I have observed, with the
GNAT compiler, what may be an undesirable consequence of this
definition: the non-composability of record types under non-default
Bit_Order. The problem may, of course, be limited to the GNAT
implementation of the standard, in which case I apologise for bothering
the ARG unnecessarily.

For background, I am working on applications that typically execute in
big-endian targets (SPARC architecture, ERC32 and LEON implementations)
but are extensively tested also in little-endian development machines
(Intel x86) and sometimes must also fully execute on such machines (for
simulation purposes). These applications typically access real or
simulated HW registers defined as record types with representation
clauses, using either High_Order_First or Low_Order_First bit-order,
depending on the convention used in the applicable documentation of the HW.

Note that as the code must run on both big-endian and little-endian
machines, whichever Bit_Order value is used in the code will be the
non-default order on some machines.

Sometimes these records have an internally repeated structure that would
be simpler to define by using a sub-record type for the repeated
structure. For example, a 32-bit register may control a UART module with
four UART links, and dedicate 7 control bits to each link. The natural
type factorisation would look like this:

    type UART_Link is record
       .. definitions of the 7 bits for one link
    end record;

    type UART_Module is record
       Link_1 : UART_Link;
       Link_2 : UART_Link;
       Link_3 : UART_Link;
       Link_4 : UART_Link;
       More   : Four_More_Bits;
    end record;

with suitable record-representation clauses and Size and Bit_Order clauses.

What I observe is that when the sub-record (UART_Link) has the
non-default bit order, GNAT will add unused padding bits to round up its
size (here 7 bits) to the nearest "natural" scalar size (here 8 bits).
This means that the components of the sub-record type (Link_1 .. Link_4)
will not fit into the bit ranges allocated to these components in the
containing record (UART_Module). Hence, the record structure cannot be
composed in this way, and all 28 bits must instead be defined directly
in the containing record (UART_Module).

Is this rounding-up of the sub-record size a consequence of the
standard's rules for non-default bit-order?

If so, could the rules be changed to avoid this rounding-up, and so
enable the composition of records with non-default bit order?

It seems to me that a record type with 7 one-bit components, each
allocated one bit position, should be able to have a size of 7 bits,
whichever bit-order is used. The positions of these bits in the
containing machine scalar should be determined later, when the record
type is used to declare an object.

***************************************************************

!topic Bit_Order for packed arrays
!reference Ada 2012 RM13.2
!from Niklas Holsti 18-01-15
!keywords Pack Bit_Order packed array representation
!discussion

Many Ada programs control HW through memory-mapped registers with more
or less complex structures, defined as record types with record
representation clauses. The programmer can control the position of each
record component and, with the Bit_Order aspect, can control whether bit
positions are numbered in increasing or decreasing significance order.

However, if a component of such a register, or the entire register, is
best described as a packed array, for example describing a 32-bit
register as "array (0 .. 31) of Boolean with Pack", the programmer
cannot specify, and cannot even directly query, how the compiler maps
array indices to bit positions. (The programmer can examine the mapping
with Unchecked_Conversions between the array type and an integer type,
but that is cumbersome.)

I suggest that it should be possible to query and/or specify the
Bit_Order aspect of a packed array type, with the same meaning and/or
effect on the representation of the array as if the array were a record
type with a component for each index.

For example, if the compiler accepts this declaration:

    type Word_Bits is array (0 .. 31) of Boolean
    with Pack, Size => 32, Bit_Order => System.Low_Order_First;

then the representation of a Word_Bits object in memory should be the
same as for this record type:

    type Word_Bits_Rec is record
       Comp_0 : Boolean;
       Comp_1 : Boolean;
       ...
       Comp_31 : Boolean;
    end record
    with Size => 32, Bit_Order => System.Low_Order_First;

    for Word_Bits_Rec use record
       Comp_0 at 0 range 0 .. 0;
       Comp_1 at 0 range 1 .. 1;
       ...
       Comp_32 at 0 range 31 .. 31;
    end record;

The recommendation would be to support the Bit_Order aspect for packed
arrays with a size not exceeding the largest machine scalar.

Slicing packed arrays with non-default bit order might be forbidden, if
seen as too hard to implement.

***************************************************************

From: Randy Brukardt
Sent: Friday, January 19, 2018  2:12 AM

...
> What I observe is that when the sub-record (UART_Link) has the
> non-default bit order, GNAT will add unused padding bits to round up
> its size (here 7 bits) to the nearest "natural"
> scalar size (here 8 bits).
> This means that the components of the sub-record type (Link_1 ..
> Link_4) will not fit into the bit ranges allocated to these components
> in the containing record (UART_Module).
> Hence, the record structure cannot be composed in this way, and all 28
> bits must instead be defined directly in the containing record
> (UART_Module).
>
> Is this rounding-up of the sub-record size a consequence of the
> standard's rules for non-default bit-order?

I don't think so. I think you are referring to the rules in 13.5.1(10.1-3/2) and
(13.1-4/2). These only put a restriction on the value of Last_Bit in the
non-default order, and make the Offset significant in the non-default order.

By the latter I mean, that in the default order
     1 at 0 .. 7;
and
     0 at 8 ..15;
mean the same thing (on typical 8-bit machines).

But in the non-default order, these may have different meanings, because the
underlying machine scalar is a byte for the first and a word for the second.

Nowhere is there any rounding-up. Gnat is well-known for rounding up sizes for
no (language) reason, so I suspect that it is either a bug or a "feature".

Did you try asking them about this case??

> If so, could the rules be changed to avoid this rounding-up, and so
> enable the composition of records with non-default bit order?
>
> It seems to me that a record type with 7 one-bit components, each
> allocated one bit position, should be able to have a size of 7 bits,
> whichever bit-order is used. The positions of these bits in the
> containing machine scalar should be determined later, when the record
> type is used to declare an object.

That's how I'd expect it to work. (Janus/Ada doesn't allow this at all;
composite objects have to be allocated on a storage unit boundary -- but this
has always been considered an unimplemented feature of Ada, it's clearly useful
in some cases. It's also terribly expensive to implement, as it seems to require
some sort of bit pointer.)

***************************************************************

From: Randy Brukardt
Sent: Friday, January 19, 2018  4:05 PM

...
> I suggest that it should be possible to query and/or specify the
> Bit_Order aspect of a packed array type, with the same meaning and/or
> effect on the representation of the array as if the array were a
> record type with a component for each index.

You seem to be suggesting a general rule that would only make sense for a packed
array of boolean whose size is that of a machine scalar. What happens for a type
that is not one of those things? If it is illegal, then defining a fairly
complex feature just for a single case (as in your example) seems like overkill.
If it is allowed, what does it mean for (say) 3-bit components? We wouldn't want
a definition that required support for packed arrays that otherwise would not be
required.

What do you need to solve your problem(s)??

> However, if a component of such a register, or the entire register, is
> best described as a packed array ...

How often does this actually happen? Since the introduction of modular types,
most components are better described as one of those rather than a packed array
(the same operations are available other than indexing/slicing -- and those are
very expensive to implement [and execute, in the slice case]). You really would
need individual access to the bits (via indexing) for it to make sense to
describe a component that way, and in that case, it's usually better to give
them names and define them as a record type.

My personal experience has been that the lack of packed arrays in Janus/Ada
(they've never been supported) is not a real problem, as most structures are
better described as a record type anyway. (The inability to put record types on
non-storage unit boundaries has been more of an issue.) But I haven't interfaced
with hardware in a number of years, so I don't put too much weight on that.

My point here being that this is a fairly complex and narrowly targeted feature
(if you don't use both bit orders in your code, it is irrelevant as well), so I
would like to know how often this would be used before insisting that every Ada
vendor figure out how to support it.

...
> Slicing packed arrays with non-default bit order might be forbidden,
> if seen as too hard to implement.

Slicing packed arrays is too hard to implement :-), but that's required anyway.

***************************************************************

From: Niklas Holsti
Sent: Tuesday, January 23, 2018  5:42 PM

> ...
>> I suggest that it should be possible to query and/or specify the
>> Bit_Order aspect of a packed array type, with the same meaning and/or
>> effect on the representation of the array as if the array were a
>> record type with a component for each index.
>
> You seem to be suggesting a general rule that would only make sense
> for a packed array of boolean whose size is that of a machine scalar.

I did not mean it to be limited to arrays of Boolean, nor to sizes equal to a
machine scalar. For example, in the example code that I submitted for the issue
"Composable records with non-default bit order" this appears:

    type Main_Record_T is record
       P  : Padding_T;
       S4 : Sub_Record_T;
       S3 : Sub_Record_T;
       S2 : Sub_Record_T;
       S1 : Sub_Record_T;
    end record;

with an associated record representation clause defining 4 bits for P and 7 bits
each for S1..S4. However, with the components S1..S4 listed in this way, the
programmer cannot iterate over the components, nor select one of the components
with an index 1..4.

This record might be more flexibly defined as:

    type Sub_Records_T is array (1 .. 4) of Sub_Record_T;
    for Sub_Records_T'Component_Size use 7;
    pragma Pack (Sub_Records_T);

    type Main_Record_T is record
       P : Padding_T;
       S : Sub_Records_T;
    end record;

with a record representation clause that defines 4 bits for P and 28 bits for S.

However, if the actual bit positions of S1..S4 are important (as for a HW
register), this form, with an array component, is currently unportable, because
the programmer does not know and cannot specify how the 28 bits allocated to the
S component are or should be divided into 4 groups for the 4 sub-components.
Should the index order 1..4 correspond to high-order first or low-order first
bit numbering?

I am suggesting that the Bit_Order should be specifiable, or at least that it
should be possible to query the Bit_Order so that the programmer could map array
indices accordingly (the Bit_Order should then be a static value, to let the
compiler optimise away the code for the unused indexing order).

The Ada record representation clauses are often quoted as an advantage over C:
the programmer does not have to use shift-and-mask to select a bit field, nor
the corresponding (but more complex) shift-and-or to set it. However, in my
project that has a HW register like Main_Record_T, the code must currently use
just such shift-mask-etc operations to access the desired component S1..S4
(which means that the record representation clause for Main_Record_T became
useless and was discarded).

> What happens for a type that is not one of those things?

I think that the analogy I gave, with a record type and representation clause
that lists the array components in index order as record components, should
apply whatever the size of the array component (up to a total size of the
largest machine scalar). The bit ranges for each component should contain
Component_Size bits, of course.

> How often does this actually happen?

I think most computers have registers where bit indexing would be useful. The
most pervasive case are the registers for controlling and querying interrupts,
usually indexed by interrupt number, but there are many others.

> Since the introduction of modular
> types, most components are better described as one of those rather
> than a packed array (the same operations are available other than
> indexing/slicing
> -- and those are very expensive to implement [and execute, in the
> slice
> case])

If both the source array and the slice are packed, and fit in a machine scalar,
it seems to me that a slice is just a shift, possibly followed by a mask. Hardly
expensive, but perhaps I am missing something.

> You really would need individual access to the bits (via indexing) for
> it to make sense to describe a component that way, and in that case,
> it's usually better to give them names and define them as a record type.

I find that there are many cases where dynamic bit indexing is desirable.

All embedded applications I have implemented in Ada have required a package with
subprograms to get and set arbitrary bit-fields of HW words, where the high and
low bit numbers are defined by subprogram parameters. Such operations are of
course easy to implement now, with shifts-mask-etc of Interfaces.Unsigned_nn
types, which is also independent of bit order, all good.

Ada has two basic kinds of composite types: records and arrays. Currently, the
representation of records can be defined (fairly) portably with respect to bit
order, but the representation of packed arrays cannot.

> My point here being that this is a fairly complex and narrowly
> targeted feature (if you don't use both bit orders in your code, it is
> irrelevant as well),

For data that connects to the external world (such as HW registers), the issue
is relevant whether the code uses one bit order or both, because we don't *know*
which bit-order a compiler will use for a packed array, without experimenting
with Unchecked_Conversions.

> so I would like to know how often this would be used before insisting
> that every Ada vendor figure out how to support it.

As to how often the need arises, most record representation clauses I write have
only scalar components. But the issue of array components has come up at one
point or another in every embedded Ada project that I remember.

As I understand it, compilers are free to reject representations they don't want
to support.

Moreover, I would be (almost) happy just with the ability to query Bit_Order for
a packed array.

Perhaps the "Bit_Order" attribute is not the most apt to use here, as there
would not be any bit numbers explicitly visible in the code (only in the
analogous, conceptual, record representation clause). Perhaps a new attribute
name should be defined, something like Component_Order or Component_Bit_Order.


***************************************************************

From: Christoph Grein
Sent: Friday, January 19, 2018  11:10 AM

>   type UART_Link is record
>      .. definitions of the 7 bits for one link
>   end record;
>
>   type UART_Module is record
>      Link_1 : UART_Link;
>      Link_2 : UART_Link;
>      Link_3 : UART_Link;
>      Link_4 : UART_Link;
>      More   : Four_More_Bits;
>   end record;
>
>with suitable record-representation clauses and Size and Bit_Order clauses.
>
>What I observe is that when the sub-record (UART_Link) has the non-default bit
>order, GNAT will add unused padding bits to round up its size (here 7 bits) to
>the nearest "natural" scalar size (here 8 bits). This means that the components
>of the sub-record type (Link_1 .. Link_4) will not fit into the bit ranges
>allocated to these components in the containing record (UART_Module). Hence,
>the record structure cannot be composed in this way, and all 28 bits must
>instead be defined directly in the containing record (UART_Module).

>Is this rounding-up of the sub-record size a consequence of the standard's
>rules for non-default bit-order?

   type UART_Link is record
      .. definitions of the 7 bits for one link
   end record;
   for UART_Link'Bit_Order use High_Order_First;

Let's say, big endian is the native bit order. Then the layout is

BE    |01234567
      |xxxxxxx    eightth bit unused.

Here, bit 0 is the first bit at a storage unit boundary |.

On a little endian machine, bits are counted in the other direction, again bit 0
is the first bit at a storage unit boundary |.

LE    76543210|   little endian numbering

Now   for UART_Link'Bit_Order use High_Order_First;  forces the compiler to use
a machine scalar with 8 bits and to count bits in nonnative order (NNO) starting
at the most significant bit

LE    76543210|   little endian numbering
      xxxxxxx |
NNO   01234567|

You can see that the first bit (0 in LE numbering, 7 in NNO), which is
necessarily at a storage unit boundary |, is unused.

So as I see it, GNAT is correct in using a padding bit; it cannot work without.

***************************************************************

From: Randy Brukardt
Sent: Friday, January 19, 2018  3:38 PM

For a stand-alone object, this discussion is surely correct. But that has
nothing to do with the bit order, there has to be a padding bit regardless of
bit order as 7 bits is less that the storage unit size.

In the case where the type is used as a component, though, the actual number of
bits shouldn't include any padding bits. Indeed, RM 13.1(7/2) says that padding
bits only exist when the size of the subtype is less than the size of the object
(not the case here).

It's not clear to me what the meaning of the bits in a 7-bit subrecord is
(regardless of the bit order), since those bit numbers do not necessarily have
any relation to the underlying machines bit numbering in the composite component
case (as composite components can be started on any bit). What is clear to me is
that such a type should have a 7-bit size, adding empty bits just to match some
bit numbering scheme makes no sense. That only should happen when a type is used
to define an object.

The fact that in an eight bit machine scalar the bits are numbered in some way
shouldn't make any difference in an object that isn't even 8 bits in size. The
size of a data type surely shouldn't be changed by the bit ordering! By your
interpretation, non-native bit ordering also forces all types to be some
multiple of the machine scalar size. If there is something in the RM that
implies that, it clearly is wrong.

In any case, changing the definition of bit ordering is not practical (the
current definition was carefully crafted to prevent byte swapping, which cannot
be allowed so as to prevent the hiding of very expensive code behind a rep
clause). So I think it is between the OP and their compiler vendor.

The only practical alternative I can see for the Standard would be to abandon
any rules altogether and make the non-native bit order implementation-defined.
In that case, I would probably have Janus/Ada do the following:

	   type UART_Link is record
	     .. definitions of the 7 bits for one link
	   end record;
	   for UART_Link'Bit_Order use High_Order_First;
---------------------------------------^
*ERROR* Wrong bit order - throw that junk hardware away and get some that can count properly. ;-)

P.S. What Janus/Ada actually does is:
	   for UART_Link'Bit_Order use High_Order_First;
---------------^
*ERROR* Feature not implemented

and there is a very low priority on changing that.

***************************************************************

From: Christoph Grein
Sent: Saturday, January 20, 2018  12:22 PM

    for UART_Link'Bit_Order use High_Order_First;

    LE    76543210|
    data  xxxxxxx |
    NNO   01234567|

    for UART_Module use record
       Link_1 use 0 range  0 ..  6;
       Link_2 use 0 range  7 .. 13;
       Link_3 use 0 range 14 .. 20;
       Link_4 use 0 range 21 .. 27;
       More use 0 range 28 .. 31:
    end record;
    for UART_Module'Bit_Order use High_Order_First;

Just guessing what this might mean on LE (didn't try it on GNAT) if no padding
should be used as you say:

    byte         3        2        1        0
    LE 10987654|32109876|54321098|76543210| counting right to left
    data 11111112|22222233|33333444|4444mmmm|
    NNO 01234567|89012345|67890123|45678901| 32 bit machine scalar

I do think this to be the only obvious or possible interpretation. I'd like to
hear what AdaCore think about this or other Ada lawyers.

As I understand Niklas he seems to say that GNAT uses a padding bit also here so
that the whole thing does not fit into 32 bits.

***************************************************************

From: Tucker Taft
Sent: Saturday, January 20, 2018  1:26 PM

> Is this rounding-up of the sub-record size a consequence of the standard's
> rules for non-default bit-order?

Very unlikely.  Can you provide the full set of record-rep, Size, and Bit_Order
clauses?  Without seeing those, it will be very hard to give you useful
information.  A compilable "reproducer" of the problem would be ideal.

***************************************************************

From: Niklas Holsti
Sent: Tuesday, January 23, 2018  2:57 AM

> ...
>> What I observe is that when the sub-record (UART_Link) has the
>> non-default bit order, GNAT will add unused padding bits to round up
>> its size (here 7 bits) to the nearest "natural"
>> scalar size (here 8 bits).
    ...
> Nowhere is there any rounding-up. Gnat is well-known for rounding up
> sizes for no (language) reason, so I suspect that it is either a bug
> or a "feature".

This seems likely, now, after I have made some more trials that I will report in
a later message.

> Did you try asking them about this case??

Not yet, partly because I wanted to get this issue on the list before the
deadline (yes, procrastination, mea culpa, mea maxima culpa). I will certainly
do so now, because it seems that there is some sort of bug in GNAT here.

I may have been hasty in posting the issue on ada-comment, if it turns out to be
just a GNAT bug, in which case I apologise for the noise I have added.

>> It seems to me that a record type with 7 one-bit components, each
>> allocated one bit position, should be able to have a size of 7 bits,
>> whichever bit-order is used. The positions of these bits in the
>> containing machine scalar should be determined later, when the record
>> type is used to declare an object.
>
> That's how I'd expect it to work.

Good to hear, and something I can use when I contact AdaCore on this question.

> (Janus/Ada doesn't allow this at all;
> composite objects have to be allocated on a storage unit boundary --
> but this has always been considered an unimplemented feature of Ada,
> it's clearly useful in some cases. It's also terribly expensive to
> implement, as it seems to require some sort of bit pointer.)

According to my recent experiments, as long as the default bit order is used,
GNAT can allocate composite components in the middle of a storage unit, at least
when the components are "small".

One can even pass such components as "in out" parameters. I think GNAT passes
them by copy-in, copy-out, which is as efficient as for a scalar component.

However, if one makes the component aliased, and tries to take its 'Access, GNAT
complains that the component must be allocated on a storage unit boundary (in my
experiments, octet boundary) and take up a whole number of storage units
(octets). So no bit pointers are allowed.

(I can't refrain from noting that several microcontroller models, including some
recent ARM Cortex models, provide a bit-addressed memory range, so bit pointers
are not always expensive.)

I will respond further in a separate message.

***************************************************************

From: Niklas Holsti
Sent: Tuesday, January 23, 2018  3:28 AM

    [snip]

> I do think this to be the only obvious or possible interpretation. I'd
> like to hear what AdaCore think about this or other Ada lawyers.

I'll be contacting AdaCore and will copy their answer to this list.

> As I understand Niklas he seems to say that GNAT uses a padding bit
> also here so that the whole thing does not fit into 32 bits.

Yes, but as will appear in a coming message from me, the padding bit is used if
the sub-record type has pragma Pack, but does not have a Size clause. I assume a
padding bit is legal in this case, as the compiler can decide how much packing
pragma Pack does.

***************************************************************

From: Niklas Holsti
Sent: Tuesday, January 23, 2018  4:05 AM

> Very unlikely.  Can you provide the full set of record-rep, Size, and
> Bit_Order clauses?  Without seeing those, it will be very hard to give
> you useful information.  A compilable "reproducer" of the problem
> would be ideal.

I have made further experiments with the GNAT versions at hand. Here is the example code. The code has several commented-out lines that represent various alternatives I have tried. I discuss my experimental results after the code.

However, and with thanks to Simon Wright who e-mailed me off-list to report his similar experiments, I now think that the example triggers a GNAT bug, and may therefore not be of interest to ada-comment. However, we will see later if AdaCore agree, and if 
they propose to correct the bug in a way that will solve my problem.

-------------------------------------- begin example with System;

package Rec_Types is

    type Sub_Record_T is record
       A : Boolean;
       B : Boolean;
       C : Boolean;
       D : Boolean;
       E : Boolean;
       F : Boolean;
       G : Boolean;
    end record;
    --
--     for Sub_Record_T'Bit_Order use System.Low_Order_First;
    for Sub_Record_T'Bit_Order use System.High_Order_First;
--     pragma Pack (Sub_Record_T);
    for Sub_Record_T'Size use 7;
    --
    for Sub_Record_T use record
       A at 0 range 6 .. 6;
       B at 0 range 5 .. 5;
       C at 0 range 4 .. 4;
       D at 0 range 3 .. 3;
       E at 0 range 2 .. 2;
       F at 0 range 1 .. 1;
       G at 0 range 0 .. 0;
    end record;

    type Padding_T is mod 2**4;

    type Main_Record_T is record
       P  : Padding_T;
       S4 : Sub_Record_T;
       S3 : Sub_Record_T;
       S2 : Sub_Record_T;
       S1 : Sub_Record_T;
    end record;
    --
    for Main_Record_T'Bit_Order use System.Low_Order_First;
--     for Main_Record_T'Bit_Order use System.High_Order_First;
    for Main_Record_T'Size use 32;
    --
    for Main_Record_T use record
       P  at 0 range 28 .. 31;
       S4 at 0 range 21 .. 27;
       S3 at 0 range 14 .. 20;
       S2 at 0 range  7 .. 13;
       S1 at 0 range  0 ..  6;
    end record;

end Rec_Types;
-------------------------------------------- end example

The two compilers I tried are:

- Native GNAT Pro 7.4.1 on linux, 32 bit
- Cross  GNAT Pro 17.0w, 32-bit linux to SPARC/LEON2

The results are as follows (similar to Simon Wright's results):

If the Sub_Record_T is defined with a record representation clause and a Size clause (stating 7 bits):

- compilation and execution work when default Bit_Order is used

- if the non-default Bit_Order is used, GNAT rejects the
   record representation clause because "bit number out of
   range of specified size". Both bit number 0 and bit number 7
   (if used instead of 0) are rejected. Bit numbers 1..6 are
   accepted.

- I suspect that GNAT is mapping the non-default-order bit numbers
   to default-order bit numbers for an 8-bit containing scalar, and
   then noting that bit number 0 maps to bit number 7, which is too
   large for a 7-bit Size.

If the Sub_Record_T is defined with a record representation clause and pragma
Pack, but without a Size clause, then:

- whichever Bit_Order is used, GNAT makes Sub_Record_T use 8 bits

- GNAT then rejects the declaration of Main_Record_T, both because
   the Sub_Record_T components don't fit into their bit fields, and
   because the specified Size of 32 bits is too small.

I will report this problem to AdaCore and summarise their reply to this list.
Thanks for all your comments, they will help my discussion with AdaCore.

***************************************************************

From: Tucker Taft
Sent: Wednesday, January 24, 2018  12:06 PM

This will work if in Sub_Record_T you specify A to G to be at bits 7 .. 1
instead of 6 .. 0.

A colleague recommended you look at AI12-0218, an "Ada Issue" that talks about
the GNAT Scalar_Storage_Order attribute, and includes a reference to an
Ada-Europe 2013 paper on the topic:

   http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ai12s/ai12-0218-1.txt?rev=1.1

Unfortunately, non-default bit order is a tricky topic, and the Ada reference
manual has defined the semantics to enable certain uses, but they don't fully
handle cases like the one you are trying.  The Scalar_Storage_Order attribute is
an attempt to address more such cases.

***************************************************************

From: Christoph Grein
Sent: Thursday, January 25, 2018  1:15 PM

> The fact that in an eight bit machine scalar the bits are numbered in
> some way shouldn't make any difference in an object that isn't even 8
> bits in size. The size of a data type surely shouldn't be changed by
> the bit ordering! By your interpretation, non-native bit ordering also
> forces all types to be some multiple of the machine scalar size. If
> there is something in the RM that implies that, it clearly is wrong.

I cannot see how this could be otherwise:

  type UART_Link is record
     A, B, C, D, E, F, G: Boolean;
   end record;
   for UART_Link use record
     A at 0 range 0 .. 0;
     B at 0 range 1 .. 1;
     C at 0 range 2 .. 2;
     D at 0 range 3 .. 3;
     E at 0 range 4 .. 4;
     F at 0 range 5 .. 5;
     G at 0 range 6 .. 6;
   end record;
   for UART_Link'Size use 7;

That's OK on the native bit order (say LE). Now non-native BE:

   for UART_Link'Bit_Order use System.High_Order_First;

GNAT then rightly flags component A: "bit number out of range of specified
size", so the type's size cannot be 7. Omitting the size clause makes the
declaration legal. The type's size must be the machine scalar's size, one cannot
omit the first bit in NNO:

-- LE    76543210|    native order
-- data  ABCDEFG |
-- NNO   01234567|    nonnative order

So I do think that NNO forces the declaration to always use a full machine
scalar.

***************************************************************

From: Randy Brukardt
Sent: Thursday, January 25, 2018  4:09 PM

> That's OK on the native bit order (say LE). Now non-native BE:
>
>    for UART_Link'Bit_Order use System.High_Order_First;
>
> GNAT then rightly flags component A: "bit number out of range of
> specified size", so the type's size cannot be 7.

This is a nonsense error message (and conclusion): the bit number of component A
is 0, that can't possibly be out of the range of the specified size, regardless
of what that is.

> Omitting
> the size clause makes the declaration legal. The type's size must be
> the machine scalar's size, one cannot omit the first bit in NNO:
>
> -- LE    76543210|    native order
> -- data  ABCDEFG |
> -- NNO   01234567|    nonnative order

You're assuming the conclusion again. The only reason that you can't "omit the
bit" is because you've assumed that the object's size is that of a full machine
scalar.

> So I do think that NNO forces the declaration to always use a full
> machine scalar.

It clearly does in GNAT, I still see no reason that that has to be the case.
GNAT rounds the sizes of all kinds of things up for no obvious reason,
regardless of the bit order.

Use of the non-native bit order always leads to madness, so my condolences to
the unlucky souls who have no choice but to do that.

Most of my career has been spent on little-endian machines (which number bits as
God and Intel intended ;-), and the short time spent compiling to big-endian
machines has been maddening. Even straightforward stuff like parameter passing
does not work(*) on a big-endian machine. Wasted months of my life. :-)

However, I have no interest in taking on the madness of non-native bit order, in
order to reduce (just a bit) the madness felt by the few unlucky souls. That
just doubles the number of mad people. :-) The proper solution is ensure that a
system uses a single bit order. (Yes, I'm dreaming.)

(*) More accurately, work sensibly. The issues can be fixed by handling
parameters specially with non-optimal instruction selection on the big-endian
machine.

***************************************************************

From: Randy Brukardt
Sent: Thursday, January 25, 2018  4:21 PM

...
> A colleague recommended you look at AI12-0218, an "Ada Issue"
> that talks about the GNAT Scalar_Storage_Order attribute, and includes
> a reference to an Ada-Europe 2013 paper on the topic:
>
> http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ai12s/ai12-0218-1.txt?rev=1
> .1

Tucker didn't mention that we decided that standardizing this is a bridge too
far for Ada 2020. We're hoping to get a Standards ready write-up from the
AdaCore people on the topic, so that other vendors could adopt it if they wish
(and actually be compatible with GNAT).

But actually implementing this requires extensive and expensive changes
thoughout the compiler (front end and back end) [because of the possibility of
non-contiguous objects]. It seems better to let vendors decide for themselves if
they need this rather than discouraging the adoption of Ada 2020 by loading
another expensive and rarely used feature into it.

If one is using GNAT, use the Scalar_Storage_Order aspect and don't worry about
portability. Odds are, that hardware-specific code will have to be reworked
anyway if one changes to a different compiler/platform. (Changing to a different
compiler without changing the hardware makes no sense as there is just too much
implementation-dependent stuff in hardware-specific runtimes.)

***************************************************************

From: Jeffrey Cousins
Sent: Thursday, January 25, 2018  4:47 PM

I generally agree with Randy's preference for little-endianism (including the
tongue in cheek bits), but note that IP uses big-endianism (but then again if
you're using a NIC and not doing the comms yourself, the NIC will take care of
the byte swapping). It seems to have settled down now, but we found the number
of bug reports for GNAT Scalar_Storage_Order a bit scary, and had to point out
in our justification for using GNAT that we didn't use Scalar_Storage_Order; it
couldn't have been easy to implement.

***************************************************************

From: Christoph Grein
Sent: Friday, January 26, 2018  2:50 AM

>>GNAT then rightly flags component A: "bit number out of range
>>of specified size", so the type's size cannot be 7.

>This is a nonsense error message (and conclusion): the bit number of
>component A is 0, that can't possibly be out of the range of the specified
>size, regardless of what that is.

So you're saying that's a GNAT bug or (less aggressive) an idiosyncratic special
interpretation of the RM?

...

>>A colleague recommended you look at AI12-0218, an "Ada Issue"
>>that talks about the GNAT Scalar_Storage_Order attribute, and
>>includes a reference to an Ada-Europe 2013 paper on the topic:
>>
>>http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ai12s/ai12-0218-1.txt?rev=1.1

>Tucker didn't mention that we decided that standardizing this is a bridge
>too far for Ada 2020. We're hoping to get a Standards ready write-up from
>the AdaCore people on the topic, so that other vendors could adopt it if
>they wish (and actually be compatible with GNAT).

Yes, I know this AI and have commented a lot on this and am still waiting for
more info what this GNAT specific attribute does. I just don't understand
Quinot's proposal. Alas I also do not have access to this paper mentioned in the
AI (it's in Springer LNCS vol. 7896, one can buy it for 30?):

>>This attribute was presented in an Ada-Europe 2013 article:
>>"Lady Ada Mediates Peace Treaty in Endianness War".

***************************************************************

From: Dirk Craeynest
Sent: Friday, January 26, 2018  5:16 AM

About the Ada-Europe 2013 paper/presentation mentioned below.

A preview of the first 2 pages of the paper is at Springer's site:
https://link.springer.com/chapter/10.1007/978-3-642-38601-5_5

The slides of the presentation are at the conference's site:
http://www.ada-europe.org/conference2013/talks/S4A%20T2%20AE2013_SSO.pdf

Finally, IIRC Springer permits both Thomas Quinot (as author) and AdaCore (as
employer) to post the full paper on their web-site. This was done for other
papers, see for example:

https://www.adacore.com/papers/exposing-uninitialized-variables-strengthening-and-extending-run-time-check

https://www.adacore.com/papers/exposing-memory-corruption-and-finding-leaks-advanced-mechanisms-in-ada

***************************************************************

From: Niklas Holsti
Sent: Friday, January 26, 2018  1:38 PM

> This will work if in Sub_Record_T you specify A to G to be at bits 7
> .. 1 instead of 6 .. 0.

Thanks for looking into this.

However, I must ask, for which meaning of "work" does that suggestion work?

Firstly, it does not work in the version of GNAT I am using. As I reported,
these GNAT versions reject both bit number 0 and bit number 7, if the
non-default Bit_Order is used together with a 7-bit Size specification. This is
probably a bug in these GNAT versions, and no doubt using bits 7 .. 1 is
accepted in newer GNAT under the non-default Bit_Order.

Secondly, however, I predict that using bits 7 .. 1 with Low_Order_First will
again force the size to 8 bits (because GNAT will add a padding bit at position
0), so it does not solve my problem, because the code is then unportable with
respect to default bit order.

It seems to me that when GNAT is placing a record-type component into a
containing record type, GNAT is willing to ignore padding bits it has added at
the /high/ end of the component-record's (stand-alone) placement in a containing
scalar, but is not willing to ignore padding bits it has added at the /low/ end
of the scalar -- the latter bits are counted in the Size of the component
record.

For example, if GNAT has mapped Sub_Record_T into the low 7 bits of an octet
(based on the combination of the Bit_Order and the record-rep clause), GNAT is
willing to a place Sub_Record_T component in any 7-bit field of a 32-bit
Main_Record_T (even crossing octet boundaries).

In contrast, if GNAT has mapped Sub_Record_T into the high 7 bits of an octet,
GNAT considers that the size is 8 bits, and a 7-bit field is not large enough.

Before the Bit_Order attribute was introduced, the way to make record
representation (almost) endianness-portable was to compute the bit numbers with
static expressions based using an endianness constant, set according to the
endianness of the machine. It seems that this old method, combined with default
Bit_Order, is the only work-around with GNAT for this problem of composing
record types.

> A colleague recommended you look at AI12-0218, an "Ada Issue" that
> talks about the GNAT Scalar_Storage_Order attribute, and include a
> reference to an Ada-Europe 2013 paper on the topic:
>
>
> http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ai12s/ai12-0218-1.txt?rev=1
> .1

Do you mean that this GNAT attribute is relevant to the issue under discussion?
I don't see how, because in the example, the whole containing record
(Main_Record_T) fits in a machine scalar, so storage-unit boundaries are not
relevant to the record representation clause -- everything is "at 0". Moreover,
all lowest-level components are single bits, so cannot cross storage-unit
boundaries.

I'm aware of this GNAT attribute, but I have not used it, partly because it is
GNAT-specific, but mainly because my applications are designed to be independent
of octet ordering, by always accessing external data octet by octet (except, of
course, for registers that must be accessed by word, for which this record
representation issue is significant but the octet order is not).

> Unfortunately, non-default bit order is a tricky topic, and the Ada
> reference manual has defined the semantics to enable certain uses, but
> they don't fully handle cases like the one you are trying.

It seems that there is some disagreement on this list as to what the RM intends
should work.

I will raise a GNAT ticket on it, in the hope of hearing more precisely why
AdaCore think that the RM specifies, or allows, the current GNAT behaviour.

***************************************************************

From: Tucker Taft
Sent: Friday, January 26, 2018  2:04 PM

I would suggest you open a ticket with AdaCore.  My comment was based on a quick
effort on my own to get it to compile, but I can believe that there are issues
in this area with other versions of GNAT.

***************************************************************

From: Niklas Holsti
Sent: Friday, January 26, 2018  2:01 PM

    [snip about Scalar_Storage_Order]

> If one is using GNAT, use the Scalar_Storage_Order aspect and don't
> worry about portability.

Or use octet-by-octet access, perhaps with Streams or the equivalent application
code, as I do.

> Odds are, that hardware-specific code will have to be reworked anyway
> if one changes to a different compiler/platform. (Changing to a
> different compiler without changing the hardware makes no sense as
> there is just too much implementation-dependent stuff in
> hardware-specific
> runtimes.)

I don't understand what you mean, there. The "runtime" usually comes with the
Ada compiler, from the Ada vendor, so its internals don't matter to the
application SW. Unless, of course, the application SW uses some non-standard
runtime API, but IME such non-standard API use is not difficult to encapsulate,
and should not much hinder a transition from one compiler to another, on the
same HW.

***************************************************************

From: Randy Brukardt
Sent: Friday, January 26, 2018  6:35 PM

> So you're saying that's a GNAT bug or (less aggressive) an
> idiosyncratic special
> interpretation of the RM?

Well, the error message is a bug by itself, regardless of anything else. Bit 0
cannot be outside of any size. Probably it comes from adjusting the bit numbers
to 7 and then checking the size, but at a very minimum the message needs to say
that happening. (Yes, bit *7* is outside of size 7, but I see no such bit in the
source code.)

As far as the actual conclusion, it's hard to say. Saying that one can't specify
objects smaller than a storage unit with the non-default bit order seems like
nonsense to me, violating the Dewar rule (the Standard does not say nonsense,
even if it literally says nonsense :-).

The original representation clause seems reasonable to me (well, other than the
7 bit size; 4 or 6 would have seemed more realistic but surely would have ended
up with the same results).

I don't see any wording that mandates your/AdaCore's interpretation that the
size has to be a full machine scalar (nothing is said about Size at all in the
non-default bit order wording). OTOH, I don't see any wording that says the
opposite, either.

***************************************************************

From: Randy Brukardt
Sent: Friday, January 26, 2018  6:40 PM

...
> > Odds are, that hardware-specific code will have to be reworked
> > anyway if one changes to a different compiler/platform. (Changing to
> > a different compiler without changing the hardware makes no sense as
> > there is just too much implementation-dependent stuff in
> > hardware-specific runtimes.)
>
> I don't understand what you mean, there. The "runtime"
> usually comes with the Ada compiler, from the Ada vendor, so its
> internals don't matter to the application SW. Unless, of course, the
> application SW uses some non-standard runtime API, but IME such
> non-standard API use is not difficult to encapsulate, and should not
> much hinder a transition from one compiler to another, on the same HW.

I was thinking about (nearly) bare machines here (like the ones Simon has been
writing about on comp.lang.ada). In such a case, the runtime has been tailored
to the hardware, and that was probably as much work as the program. Redoing that
is impractical unless you have to do so in any case.

If, instead, you are running on an RTOS, then you may be right, if you can
actually find another vendor supporting that RTOS. Still wouldn't expect it very
often, since it would be hard to be completely independent of
implementation-dependent stuff. (Even stuff like the size of Integer can bite.)

***************************************************************

From: Niklas Holsti
Sent: Saturday, January 27, 2018  3:58 AM

> I was thinking about (nearly) bare machines here (like the ones Simon
> has been writing about on comp.lang.ada). In such a case, the runtime
> has been tailored to the hardware, and that was probably as much work as the
> program.

If Simon is implementing his own Ada runtimes, to support Ada tasking, there
will be hardware-dependent code, and also code that depends on the
compiler-specific interface between the runtime and the code generated by the
compiler for its own runtime's API. In that case I agree, it would be a lot of
work for Simon to switch compilers.

Another case where I agree is if one has to pay the Ada vendor to tailor their
off-the-shelf runtime to one's hardware. Then one does not want to repeat the
payment to a new Ada vendor, even if the payment is much less than the total
cost of the application implementation.

But if both Ada vendors, for the same hardware, provide standards-compliant
runtimes (e.g. Ravenscar), then I don't believe that changing the compiler is so
difficult. As you say, the size of Integer can change (don't use Integer...),
and more seriously, the size of the largest supported integer type may change
(say, from 64 to 32 bits).

I have observed almost such a case, where the basic framework of the on-board SW
for a satellite (the ESA GOCE satellite), implemented with the XGC compiler and
XGC Ravenscar runtime, was reused for a new project implemented on the same
processor architecture (SPARC) with GNAT and the AdaCore Ravenscar runtime. The
changes required in the application SW were small, certainly much, much smaller
than the effort to implement the application. Most of the changes were in the
Death Report procedure that is activated in case of a fatal SW failure and digs
into the runtime tasking structures to report the state of all application
tasks, before resetting the system.

***************************************************************

From: Simon Wright
Sent: Tuesday, January 30, 2018  11:26 AM

[...]

> I was thinking about (nearly) bare machines here (like the ones Simon
> has been writing about on comp.lang.ada). In such a case, the runtime
> has been tailored to the hardware, and that was probably as much work as the
> program.

For clarity, and assuming I'm the Simon concerned, the RTS[1] runs over
FreeRTOS[2]. I don't think this is unreasonable, given that the last project I
was paid to work on ran on GNAT over VxWorks.

I wrote up something about the process in [3] - skip down to "RTS: second
stage".

[1] https://github.com/simonjwright/cortex-gnat-rts
[2] https://www.freertos.org
[3] http://forward-in-code.blogspot.co.uk/2015/06/building-runtime-system-for-arm-eabi.html

***************************************************************

From: Niklas Holsti
Sent: Tuesday, January 30, 2018  4:02 PM

> For clarity, and assuming I'm the Simon concerned,

I think you are that Simon, yes.

> the RTS[1] runs over FreeRTOS[2]. I don't think this is unreasonable,
> given that the last project I was paid to work on ran on GNAT over VxWorks.

Sorry, but I have lost the thread of this sub-discussion: what is not
"unreasonable"?

Do you agree with Randy's estimate that creating or tailoring the runtime was as
much work as the (application) program?

And that therefore one would seldom change an application from one compiler to
another, without also changing the hardware? Even if the runtime is supplied by
the Ada vendors, and is not build-your-own, as in your case?

Or do you feel that the work you did, in tailoring the runtime, was not
unreasonably large?

> I wrote up something about the process in [3] - skip down to "RTS: second
> stage".

Thanks for documenting this! I believe it is a very valuable resource.

***************************************************************

Questions? Ask the ACAA Technical Agent