!standard E.1(5) 20-03-12 AI12-0359-1/02 !standard E.2.1(11) !standard E.4(1) !class ramification 20-02-04 !status Amendment 1-2012 20-03-11 !status WG9 Approved 22-06-22 !status ARG Approved 14-0-0 20-03-11 !status work item 20-02-04 !status received 19-11-18 !priority Low !difficulty Easy !subject Calls to subprograms declared in shared passive units !summary Calls to subprograms in a passive partition from an active partition are not remote calls. !question The visible part of the declaration of a shared passive package may contain subprogram, protected objects, and tasks. Can these be called directly from other partitions than the one where the shared passive unit is configured? (Yes.) !response Passive partitions can be mapped to storage nodes that have no execution resources by E.1(4). E.4(1) says that a remote subprogram call is executed by the called partition. Since a passive partition may not have any execution resources, it cannot execute anything and thus it cannot be a called partition. (The first sentence of E.4(1) should make this clearer by including the word "active"). Therefore, calls to subprograms in a passive partition are not remote calls. It is intended that the code and data of passive partitions are directly available to active partitions. It has been suggested that the fact that E.2.1(11) says that the units of a passive partition are not "needed" implies that the code cannot be included or accessed from an active partition other than one of the processing node that it is configured for. This is not true: "needed" here simply means that the code and (most importantly) data are not part of the active partition. It does not prevent duplicating the code of a passive partition in each active partition that uses it (obviously, the data must not be duplicated); since code doesn't have side-effects, it can be duplicated anywhere that an implementation wants. (This is an "as-if" optimization, it's not possible to tell the difference.) !wording Add an AARM Note after E.1(5): The configuration of the partitions of a program onto a distributed system shall be consistent with the possibility for data references or calls between the partitions implied by their semantic dependences. Any reference to data or call of a subprogram across partitions is called a *remote access*. {AARM Ramification: A passive partition has no execution resources of its own, so while a call of a subprogram in a passive partition is a remote access to that subprogram, it is *not* a remote subprogram call (see E.4). The calling active partition executes the body of the subprogram of the passive partition.} [Editor's note: Tucker suggested adding "active" into the wording above, but that would seem to leave accesses to passive partitions in limbo; accesses to the data in particular are certainly a form of remote access. I thought it better to differentiate between "remote access" and "remote subprogram call" (which is a subset of the former).] Add AARM Notes after E.2.1(11): Notwithstanding the rule given in 10.2, a compilation unit in a given partition does not need (in the sense of 10.2) the shared passive library units on which it depends semantically to be included in that same partition; they will typically reside in separate passive partitions. {AARM To Be Honest: This rule is necessary so that the shared data of a shared passive partition is not duplicated in each partition. It does not imply a particular implementation; in particular, code can be replicated in each active partition.} {AARM Implementation Note: One possible implementation for a shared passive partition is as a shared library that is mapped into the address space of of each active partition. In such case, both the code and the data of the passive partition would be mapped into the active partitions and directly called/accessed from each active partition. For instance, on Microsoft Windows a DLL has the correct semantics. Alternatively, the shared data can be represented as a file or persistent memory, with the shared code being replicated in each active partition. Code replication is an as-if optimization; it should be impossible to tell where the code lives since no elaboration is necessary. End AARM Implementation Note.} Modify E.4(1): A *remote subprogram call* is a subprogram call that invokes the execution of a subprogram in another {(active)} partition. The partition that originates the remote subprogram call is the *calling partition*, and the partition that executes the corresponding subprogram body is the *called partition*. Some remote procedure calls are allowed to return prior to the completion of subprogram execution. These are called *asynchronous remote procedure calls*. {AARM Discussion: Remote subprogram calls are always between active partitions; a passive partition has no execution resources of its own and thus cannot execute anything, while a remote subprogram call is always executed by the called partition.} !discussion The model of a passive partition is that the space is mapped into the address space of all active partitions to which they are connected. Remote calls are about communication between active partitions. Normal calls, as well as protected-subprogram calls, are legal between an active partition and the passive partitions with which it is connected. The various limitations associated with passive partitions are mainly to allow them to persist beyond the lifetime of any single active partition. So you can think of a passive partition as a persistent, shared, address space, visible to one or more active partitions, which might come and go while the passive partition lives on. One way to implement this might be by using a file mapping mechanism, where the passive partition's data is represented by the contents of the file. Another approach is to treat them as a shared library in the sense of a DLL on Windows (DLLs persist until all partitions using them terminate). In the file case, the code could be replicated in each active partition, with just the data shared and persistent. In the DLL case, the code would live with the shared data. Calls are made to a DLL using the caller's stack; no marshalling is needed or expected. !corrigendum E.4(1) @drepl A @i is a subprogram call that invokes the execution of a subprogram in another partition. The partition that originates the remote subprogram call is the @i, and the partition that executes the corresponding subprogram body is the @i. Some remote procedure calls are allowed to return prior to the completion of subprogram execution. These are called @i. @dby A @i is a subprogram call that invokes the execution of a subprogram in another (active) partition. The partition that originates the remote subprogram call is the @i, and the partition that executes the corresponding subprogram body is the @i. Some remote procedure calls are allowed to return prior to the completion of subprogram execution. These are called @i. !ASIS No ASIS effect. !ACATS test No additional ACATS tests are needed. CXE2001 contains such a call, so the case in question is already tested. !appendix !topic Legality of calls to subprograms declared in shared passive units !reference Ada 2012 RM E.1(5), E.2.1(11), E.4 !from Thomas Quinot 2019-11-18 !keywords distributed systems, remote calls, shared passive !discussion The visible part of the declaration of a shared passive package may contain subprogram, protected objects, and tasks. However, these cannot be called directly from other partitions than the one where the shared passive unit is configured. This should be made clearer in the RM. Per E.2.1(10) a shared passive unit may be assigned to at most one partition. Per E.2.1(11), a shared passive unit is not needed on partitions where it is not assigned. This has the immediate consequence that calls to subprograms declared in a shared passive package from a partition other than the one where the package is configured cannot be local calls, because this would require the unit to be needed on the calling partition. Also, these cannot be legal remote calls either, because even though they would constitue remote access per E.1(5), they do not fall in any of the limitative list of legal remote call cases (from E.4(2..5)). It follows that such calls can neither be local nor remote, and must be prohibited by post-compilation rules. The suggestion here is to add a clarifying item uner E.2.1 post-compilation rules stating explicitly that no unit needed in a partition may make calls to a subprogram from a shared passive unit unless that shared passive is configured on that partition. This is a ramification of the rules mentioned above. Note that this requires adjusting ACATS CXE2001, which does contain such a call: -- SPECIAL REQUIREMENTS: -- Compile the compilation units in this file. -- Create the two partitions (A and B) with the following contents: -- Partition A contains: -- CXE2001_A (main procedure) -- and all normal and pure packages with'ed by this unit. -- Partition B contains: -- CXE2001_B (main procedure) -- CXE2001_Part_B (RCI package) -- CXE2001_Shared (shared data package) -- and all normal and pure packages with'ed by these units. CXE2001_A contains direct calls to CXE2001_Shared.Shared_Counter.Increment that must be illegal per the above analysis. **************************************************************** From: Tucker Taft Sent: Monday, November 18, 2019 8:40 AM > Per E.2.1(10) a shared passive unit may be assigned to at most one > partition. Per E.2.1(11), a shared passive unit is not needed on > partitions where it is not assigned. This has the immediate > consequence that calls to subprograms declared in a shared passive > package from a partition other than the one where the package is > configured cannot be local calls, because this would require the unit > to be needed on the calling partition. If the RM says this, it is misleading. Calls to passive partitions are presumed to use non-remote calling mechanisms because the passive partition is presumed to be mapped into the address space of the active partitions that call into it. I would be curious where you found a rule that says a call to a subprogram cannot be "local" if there is no "needs" relationship to the package in which the subprogram is defined. In fact, I would be curious where you found the definition of what it means to be a "local" call. I think we mention local calls in an implementation note about how one might implement remote calls, but I don't believe we define the term formally. > Also, these cannot be legal remote calls either, because even though > they would constitue remote access per E.1(5), they do not fall in any > of the limitative list of legal remote call cases (from E.4(2..5)). They are not remote calls. > It follows that such calls can neither be local nor remote, and must > be prohibited by post-compilation rules. I don't follow the logic that they cannot be "local" calls, however that is defined. > The suggestion here is to add > a clarifying item uner E.2.1 post-compilation rules stating explicitly > that no unit needed in a partition may make calls to a subprogram from > a shared passive unit unless that shared passive is configured on that > partition. This is a ramification of the rules mentioned above. Actually, it sounds like we should clarify that non-remote calls are permitted to passive partitions. **************************************************************** From: Thomas Quinot Sent: Monday, November 18, 2019 11:31 AM > If the RM says this, it is misleading. Calls to passive partitions are > presumed to use non-remote calling mechanisms because the passive partition > is presumed to be mapped into the address space of the active partitions > that call into it. I would be curious where you found a rule that says a > call to a subprogram cannot be "local" if there is no "needs" relationship > to the package in which the subprogram is defined. In fact, I would be > curious where you found the definition of what it means to be a "local" > call. I think we mention local calls in an implementation note about how > one might implement remote calls, but I don't believe we define the term > formally. There is indeed no explicit mention of what a local call is, but my common sense interpretation of 10.2(2) is that the "need" relationship defines what code is linked into a partition so that you can call it, and conversely the fact that a given unit is not needed in a partition means that its code (and the need-closure of its body) are not present on the partition. > > Also, these cannot be legal remote calls either, because even though > > they would constitue remote access per E.1(5), they do not fall in > > any of the limitative list of legal remote call cases (from > > E.4(2..5)). > > They are not remote calls. I agree that such calls definitely are not remote calls, but I content that they cannot be local calls either [on a partition other than the one on which the SP unit is configured] since the code of the shared passive subprogram cannot be present on the partition. > > It follows that such calls can neither be local nor remote, and must > > be prohibited by post-compilation rules. > > I don't follow the logic that they cannot be "local" calls, however that is > defined. They cannot be local because in order to be able to make a local call to a subprogram, the body of that subprogram must be locally available in the calling partition, and my understanding is that the bodies available in a partition are those from "needed" library units. > > The suggestion here is to add > > a clarifying item uner E.2.1 post-compilation rules stating > > explicitly that no unit needed in a partition may make calls to a > > subprogram from a shared passive unit unless that shared passive is > > configured on that partition. This is a ramification of the rules > > mentioned above. > > Actually, it sounds like we should clarify that non-remote calls are > permitted to passive partitions. I do not think it makes any sense to say that some calls from one partition to another are not remote. **************************************************************** From: Tucker Taft Sent: Monday, November 18, 2019 12:03 PM > ... > I do not think it makes any sense to say that some calls from one > partition to another are not remote. The model of a passive partition is that the space is mapped into the address space of all active partitions to which they are connected. Without that, they don't really make sense. Remote calls are about communication between active partitions. Normal calls, as well as protected-subprogram calls, are legal between an active partition and the passive partitions with which it is connected. The various limitations associated with passive partitions are mainly to allow them to persist beyond the lifetime of any single active partition. So you can think of a passive partition as a persistent, shared, address space, visible to one or more active partitions, which might come and go while the passive partition lives on. One way to implement this might be by using a file mapping mechanism, where the passive partition's data is represented by the contents of the file. In fact, my understanding is that that is the way they has been implemented by GNAT in the past. **************************************************************** From: Tucker Taft Sent: Monday, November 18, 2019 12:06 PM This is perhaps clarified by paragraph 2 of Annex E on Distributed Systems: "A distributed system is an interconnection of one or more processing nodes (a system resource that has both computational and storage capabilities), and zero or more storage nodes (a system resource that has only storage capabilities, with the storage addressable by one or more processing nodes)." The key word is "addressable" meaning mapped directly into the address space. **************************************************************** From: Randy Brukardt Sent: Monday, November 18, 2019 6:54 PM This implies that the *data* of a shared passive package is mapped into the address space. But Thomas's question is about any *code* associated with a shared passive package. I think the model is intended to be that the code is duplicated in each partition, with the data shared between all of the partitions. At least, that's how I read the above. I can't reconcile this model with E.2.1(11): of course the partition "need"s the shared passive library units -- how else is the code going to be linked with each partition? If the code was *really* in a separate partition, then it would need to be called remotely, and I think we all agree that doesn't happen for shared passive partitions. I note that Pure packages are duplicated in each partition (the code should be the same, and there is no state), and the same should be true for a shared passive package (the state is managed separately from any partition, but the code referencing that state can be duplicated for each partition). So at the least, E.2.1(11) is very confusing, since it is referring to a hypothetical model which is impractical in practice (mapping of code space), and seems to vary from the model described in the introduction as quoted above. At a minimum, we ought to add some AARM notes to explain this further, perhaps an AARM Implementation Note to explain the difference between data and code for these units. (It would make sense to describe the implementation model of Pure packages, too, in such a note.) **************************************************************** From: Tucker Taft Sent: Monday, November 18, 2019 8:32 PM > But Thomas's question is about any *code* associated with a shared > passive package. Does it really matter? It could be in the shared passive address space as well. Or in some shared area. The key point is there is no elaboration required, so the code can live anywhere -- it could be mapped into the address space of any active partition that needs it, or it could be linked in. I agree we should clarify, but I don't think the manual actually makes any real statement about where code lives. > I think the model is intended to be that the code is duplicated in > each partition, with the data shared between all of the partitions. At > least, that's how I read the above. It might be shared, or it might be duplicated. I don't the manual really cares. > I can't reconcile this model with E.2.1(11): of course the partition > "need"s the shared passive library units -- how else is the code going > to be linked with each partition? If the code was *really* in a > separate partition, then it would need to be called remotely, and I > think we all agree that doesn't happen for shared passive partitions. The code could be in a "separate" *passive* partition without requiring a remote call, so long as the code is addressable. > I note that Pure packages are duplicated in each partition (the code > should be the same, and there is no state), and the same should be > true for a shared passive package (the state is managed separately > from any partition, but the code referencing that state can be duplicated > for each partition). Agreed, but it need not be duplicated, since the passive partition's data is being mapped into multiple process' address spaces, there is no particular reason that the code could not be similarly mapped into multiple process' code space. > So at the least, E.2.1(11) is very confusing, since it is referring to > a hypothetical model which is impractical in practice (mapping of code > space), Not sure why you say it is impractical. There are many systems where the code of the operating system is mapped into every process' space. There are also systems where shared code modules are given unique addresses in the huge 64-bit address space, and every process that needs them maps them into their address space. This is like a DLL, only once it has been loaded into memory once, it never needs to be loaded again. > and seems to vary from the model described in the introduction as > quoted above. At a minimum, we ought to add some AARM notes to explain > this further, perhaps an AARM Implementation Note to explain the > difference between data and code for these units. (It would make sense > to describe the implementation model of Pure packages, too, in such a > note.) I agree we should clarify, but I am not convinced there is any inconsistency in the manual as written. It sounds like the model of shared code is unfamiliar enough that we need to talk about it explicitly, at least in an AARM note. **************************************************************** From: Randy Brukardt Sent: Monday, November 18, 2019 9:01 PM > > But Thomas's question is about any *code* associated with a shared > > passive package. > > Does it really matter? It could be in the shared passive address > space as well. Or in some shared area. The key point is there is no > elaboration required, so the code can live anywhere -- it could be > mapped into the address space of any active partition that needs it, > or it could be linked in. It matters if only because the introduction you quoted above clearly has a different model. Note that it talks about "storage nodes". The only construct in Annex E that could be used to describe such a node is a shared passive package. (The others are either active or have no state to share as in a Pure package.) But then we have to describe where the code associated with accessing a "storage node" lives. It surely can't live on the storage node (no "computational capabilities" as described above). And E.2.1(11) implies that it can't live on the processing nodes, either. ... > I agree we should clarify, but I don't think the manual actually makes > any real statement about where code lives. Agreed. It's just confusing, since it *appears* to require something unlikely (the code living with the storage). ... > > I can't reconcile this model with E.2.1(11): of course the partition > > "need"s the shared passive library units -- how else is the code > > going to be linked with each partition? If the code was *really* in > > a separate partition, then it would need to be called remotely, and > > I think we all agree that doesn't happen for shared passive partitions. > > The code could be in a "separate" *passive* partition without > requiring a remote call, so long as the code is addressable. OK, but the above appears to ban a duplicated implementation. > > I note that Pure packages are duplicated in each partition (the code > > should be the same, and there is no state), and the same should be > > true for a shared passive package (the state is managed separately > > from any partition, but the code referencing that state can > be duplicated for each partition). > > Agreed, but it need not be duplicated, since the passive partition's > data is being mapped into multiple process' > address spaces, there is no particular reason that the code could not > be similarly mapped into multiple process' code space. Because it's inefficient to do so? > > So at the least, E.2.1(11) is very confusing, since it is referring > > to a hypothetical model which is impractical in practice (mapping of > > code space), > > Not sure why you say it is impractical. There are many systems where > the code of the operating system is mapped into every process' space. > There are also systems where shared code modules are given unique > addresses in the huge 64-bit address space, and every process that > needs them maps them into their address space. This is like a DLL, > only once it has been loaded into memory once, it never needs to be > loaded again. Sure, it's possible, but I said "impractical" for a reason. Calling code in a DLL needs a special calling sequence (using dynamic lookups and an indirect call through the result), it needs special code generation (to avoid depending on a non-existent runtime), and so on. It surely could be done if someone wanted to do so badly enough. But the GNAT implementation uses a file-store to emulate the partition's storage, and any implementation like that (which is pretty much the only one I would expect anyone to want) would want any code to be duplicated (it surely doesn't want the code in the file-store). One could generate the code into a DLL or the similar for other systems, but that would be very hard work compared to simply linking the needed units into the partition. (Those units would be compiled to use a file-store, but otherwise would be pretty similar to any other unit.) The problem here is that E.2.1(11) appears to ban such a simple and obvious implementation. That is my reading of Thomas's question. I tend to apply the Dewar rule here: the RM does not say nonsense, and E.2.1(11) is nonsense. It really tried to say something about the state being shared, but since there's no term for that, it got off into madness. > > and seems to vary from the model described in the introduction as > > quoted above. At a minimum, we ought to add some AARM notes to > > explain this further, perhaps an AARM Implementation Note to explain > > the difference between data and code for these units. (It would make > > sense to describe the implementation model of Pure packages, too, > > in such a note.) > > I agree we should clarify, but I am not convinced there is any > inconsistency in the manual as written. It sounds like the model of > shared code is unfamiliar enough that we need to talk about it > explicitly, at least in an AARM note. See above. The introduction you quoted (non-normative) clearly states that some nodes don't support code. Yet E.2.1(11) requires the code to be (logically) part of a storage node - there's no leeway to put it elsewhere. I agree of course that an as-if optimization is possible to duplicate the code anyway -- but that's something that the RM really should talk about. It's really easy to interpret E.2.1(11) as Thomas did -- essentially banning putting the code elsewhere than the "shared passive partition". If that's not the intent (and I think we agree that it's not the intent), then we ought to say so somewhere (other than a non-normative introductory paragraph). **************************************************************** From: Jean-Pierre Rosen Sent: Tuesday, November 19, 2019 2:19 AM > But Thomas's question is about any *code* associated with a shared > passive package. > > I think the model is intended to be that the code is duplicated in > each partition, with the data shared between all of the partitions. At > least, that's how I read the above. Well, the distributed model is not only about networks. The notion of shared passive is better understood if you think of several boards plugged into a back-plane. An active partition is a board with a CPU. A passive partition is a board without a CPU, but that can hold RAM (variables) and ROM (subprograms and constants). Of course, that RAM/ROM must be addressable from active partitions, but a call to a subprogram is just a regular call to a subprogram that is part of the address space of the active partition. I think shared-passive were really intended for that use case. Gnat managed to implement shared-passive over a network by putting RAM into shared files, and by duplicating ROM (which is allowed by as-if rule). But it's kind of a stretch... **************************************************************** From: Thomas Quinot Sent: Tuesday, November 19, 2019 5:15 AM > Does it really matter? It could be in the shared passive address space as > well. Or in some shared area. The key point is there is no elaboration > required, so the code can live anywhere -- it could be mapped into the > address space of any active partition that needs it, or it could be linked > in. > > I agree we should clarify, but I don't think the manual actually makes any > real statement about where code lives. I beg to disagree. My understanding is that the need relationship precisely is about where code lives. > > I think the model is intended to be that the code is duplicated in > > each partition, with the data shared between all of the partitions. > > At least, that's how I read the above. > > It might be shared, or it might be duplicated. I don't the manual really > cares. E.2.1(10) seems to pretty clearly exclude duplication. > > I can't reconcile this model with E.2.1(11): of course the partition > > "need"s the shared passive library units -- how else is the code > > going to be linked with each partition? If the code was *really* in > > a separate partition, then it would need to be called remotely, and > > I think we all agree that doesn't happen for shared passive partitions. > > The code could be in a "separate" *passive* partition without requiring a > remote call, so long as the code is addressable. I don't think that it is reasonable to support an interpretation of the RM whereby calling code that resides on another partition is not a remote call. > Agreed, but it need not be duplicated, since the passive partition's data is > being mapped into multiple process' address spaces, there is no particular > reason that the code could not be similarly mapped into multiple process' > code space. I have always understood the "need" relationship as defining what code is mapped in a[n active] partition's address space (i.e. what is linked in the executable for the partition). I don't think it makes sense for code to be addressable but not needed. Also, there's the issue of hidden state. Pure units can be duplicated on multiple partitions without compromising consistency because they have no state. That's not the case for shared passives, you can have a variable in an SP body. > I agree we should clarify, but I am not convinced there is any inconsistency > in the manual as written. It sounds like the model of shared code is > unfamiliar enough that we need to talk about it explicitly, at least in an > AARM note. If you mean that the RM requires an execution model where code is not needed but is adressable anyway, then all I can say is that this will require a significant amount of clarification, and this will also make the specification incompatible with the only extant implementation of annex E. **************************************************************** From: Jean-Pierre Rosen Sent: Tuesday, November 19, 2019 6:11 AM > I don't think that it is reasonable to support an interpretation of > the RM whereby calling code that resides on another partition is not a > remote call. Of course it is! Remember that this is all about /shared/ passive packages. Since they are shared, they can be directly accessed. Remote calls are about RPC, where a request is carried over a network. **************************************************************** From: Thomas Quinot Sent: Tuesday, November 19, 2019 6:28 AM Well, I disagree. For one thing, there is no mention of a network in the RM. Second, there are pretty definitive statements: * E.1(5): "Any reference to data or call fo a subprogram across partitions is called a remote access". * E.4(1): "A remote subprogram call is a subprogram call that invokes the execution of a subprogram in another partition." I really do not think it is wise to stretch the model to accomodate direct/local calls to something that is not in the calling partition. This is not only very counter-intuitive (and would require substantial clarification in the RM if this were to be the chosen interpretation in the end). This also makes the annex effectively unimplementatble except for the specific corner case of nodes that share address space. **************************************************************** From: Jean-Pierre Rosen Sent: Tuesday, November 19, 2019 10:28 AM > Well, I disagree. For one thing, there is no mention of a network in > the RM. Second, there are pretty definitive statements: > * E.1(5): "Any reference to data or call fo a subprogram across partitions > is called a remote access". A call can be a remote /access/. So? > * E.4(1): "A remote subprogram call is a subprogram call that invokes the > execution of a subprogram in another partition." Well, the next sentence says: "The partition that originates the remote subprogram call is the calling partition, and the partition that executes the corresponding subprogram body is the called partition. " A passive partition doesn't execute anything, so the first sentence may be a bit too general, but it clearly doesn't apply to subprograms in a "storage node" (see E.2) > I really do not think it is wise to stretch the model to accomodate > direct/local calls to something that is not in the calling partition. I really do not think that it is stretching the model, the model of the "shared memory board" is really how I understood passive partitions ever since Anthony Gargaro presented it for the first time! (Note for old-timers: wasn't it in Switzerland?) **************************************************************** From: Richard Wai Sent: Tuesday, November 19, 2019 11:01 AM I think it might be helpful to view the Annex E from the broader ARM perspective. The Ada Standard strives as far as possible to not specify implementation details, rather to specify behavior and specific rules. This is fairly unconventional among the more main-stream languages, especially when it comes to the coupling of languages with the larger environment (OS). C for example is fairly UNIX-oriented. I think the real source of confusion when it comes to Annex E really comes from the RPC approached popularized by most UNIX-like operating systems. It is an idea that one would essentially pass a set of parameters to another process to invoke the process to execute a procedure with those transmitted parameters, and possibly return a result. Annex E does not specify that behavior, and it tries to avoid requiring that paradigm. And in fact, the RPC paradigm does not allow for a share passive partition. An RPC call is basically like an Ada task rendezvous - one process is requesting that a separate process executes some code. But a passive partition cannot execute anything. So making a remote call to a passive partition would be in violation of the standard. It therefore stands to reason that a call to a subprogram in a shared passive partition would have to be executed by a task in some active partition. And since there does not yet exist a machine which can execute remote code in common use, the code for (and call to) that subprogram must be local to the calling (active) partition. It seems to me that the point of a shared passive partition is to allow the implementation freedom in figuring out the details of how shared data would be marshaled between separate nodes, while presenting familiar behavior to the programmer. Again, this is unlike traditional RPC, where the programmer has to be much more explicit when crossing the boundary between partitions. **************************************************************** From: Tucker Taft Sent: Tuesday, November 19, 2019 11:12 AM >>> I don't think that it is reasonable to support an interpretation of >>> the RM whereby calling code that resides on another partition is not >>> a remote call. >> Of course it is! Remember that this is all about /shared/ passive >> packages. Since they are shared, they can be directly accessed. >> Remote calls are about RPC, where a request is carried over a network. > > Well, I disagree. For one thing, there is no mention of a network in > the RM. Second, there are pretty definitive statements: > * E.1(5): "Any reference to data or call fo a subprogram across partitions > is called a remote access". > * E.4(1): "A remote subprogram call is a subprogram call that invokes the > execution of a subprogram in another partition." In both of these cases, they should have said "active partition." Shared-passive partitions are very different beasts, and alas, we probably were sloppy in sometimes saying simply "partition" when we really meant "active partition." I can see why you could reach the conclusion you have reached, but it is definitely not the original intent of the feature. We are not trying to "stretch" the model, merely trying to clarify the model, which clearly is somewhat ambiguous, given your reading of the RM. **************************************************************** From: Thomas Quinot Sent: Tuesday, November 19, 2019 11:29 AM > > * E.4(1): "A remote subprogram call is a subprogram call that invokes > > the execution of a subprogram in another partition." > Well, the next sentence says: > "The partition that originates the remote subprogram call is the > calling partition, and the partition that executes the corresponding > subprogram body is the called partition. " > > A passive partition doesn't execute anything, so the first sentence > may be a bit too general, but it clearly doesn't apply to subprograms > in a "storage node" (see E.2) I don't think this interpretation is viable. The body of a subprogram declared in a shared passive unit is not elaborated on any active partition other than the possible one where it is configured, as a consequence of (a) the body not being needed (E.2.1(11)) and (b) the program execution process (10.2(13) and 10.2(2)). And such elaboration is required to establish the possibility of calling the subprogram, per 6.3(6), regardless of the preelaborated status of the unit. And conversely if we were to take the radical standpoint that the elaboration/elaborated state of the SP unit is shared across all partitions, and that the body is therefore callable from all partitions, then E.2.1(10) (forbidding the configuration of a shared passive on more than one partition) and E.2.1(11) (excluding shared passives from the need-closure) cease to make any sense at all, as configuring a shared passive on a partition or not, and considering it as needed or not, would have exactly zero effect. **************************************************************** From: Thomas Quinot Sent: Tuesday, November 19, 2019 11:36 AM > In both of these cases, they should have said "active partition." > Shared-passive partitions are very different beasts, Well, that in itself is muddling the discussion. Not sure what you are referring to when you say "shared passive partition". There are shared passive *units*, which can be assigned to *active partitions* or *passive partitions*. Maybe the foundamental questions are: * what is the intended effect of assigning a shared passive unit to a passive partition; and * what is the intended effect of assigning a shared passive unit to an active partition? **************************************************************** From: Tucker Taft Sent: Tuesday, November 19, 2019 11:46 AM > Well, that in itself is muddling the discussion. Not sure what you are > referring to when you say "shared passive partition". There are shared > passive *units*, which can be assigned to *active partitions* or > *passive partitions*. Maybe the foundamental questions are: > * what is the intended effect of assigning a shared passive unit > to a passive partition; and It is available to all active partitions connected to the passive partition. > * what is the intended effect of assigning a shared passive unit > to an active partition? It is only available to units within that active partition. **************************************************************** From: Randy Brukardt Sent: Tuesday, November 19, 2019 3:22 PM ... > > A passive partition doesn't execute anything, so the first sentence > > may be a bit too general, but it clearly doesn't apply to > > subprograms in a "storage node" (see E.2) > > I don't think this interpretation is viable. The body of a subprogram > declared in a shared passive unit is not elaborated on any active > partition other than the possible one where it is configured, as a > consequence of (a) the body not being needed (E.2.1(11)) and > (b) the program execution process (10.2(13) and 10.2(2)). And such > elaboration is required to establish the possibility of calling the > subprogram, per 6.3(6), regardless of the preelaborated status of the > unit. > > And conversely if we were to take the radical standpoint that the > elaboration/elaborated state of the SP unit is shared across all > partitions, I don't find this as radical -- it's the only sane interpretation. In implementation terms, all of the data ("the state") is shared and all of the code is duplicated. The elaboration state (at least logically) is part of the data (it's surely part of the state), not the code. Practically, it doesn't make any difference since preelaborable code shouldn't require much if any elaboration anyway. > ... and that the body is therefore > callable from all partitions, then E.2.1(10) (forbidding the > configuration of a shared passive on more than one partition) and > E.2.1(11) (excluding shared passives from the > need-closure) cease to make any sense at all, as configuring a shared > passive on a partition or not, and considering it as needed or not, > would have exactly zero effect. Yes, that's the only sane interpretation. Clearly the Dewar rule that the RM does not say nonsense has to be applied to these paragraphs. And the only sensible thing to do with them is ignore them (with two unimportant exceptions: for E.2.1(10), this seems to allow the management of the shared store to belong to the "configured" partition, which avoids the problem of figuring out how that gets accomplished when no code can be executed; and for E.2.1(11), this says that the elaboration (which has to do nothing anyway) occurs only once). But it is clear that E.2.1(10) and E.2.1(11) have nothing useful to say about the actual organization of the code and data for a passive partition (and are confusing because it appears otherwise). Note that while an "address mapping" implementation is possible, it doesn't make much sense in practice as it would require the shared passive partition to be accessible only from a single physical processor. That's contrary to the intended usage of active partitions and would seem to be a massive restriction on the use of passive partitions. Ergo, the only sane implementation is duplicated code, shared data. And the Dewar rule has to be applied to make sense of E.2.1(11) in particular -- while the unit is not needed (and thus not elaborated), the code still has to be loaded as if it is needed. The hair-splitting necessary is easy to misunderstand. **************************************************************** From: Tucker Taft Sent: Tuesday, November 19, 2019 3:36 PM Our conclusions are probably the same but I don't understand your last paragraph. You are saying that address mapping implies that it would be accessible only from a single processor. I don't understand that. When physical memory is shared between physical processors, it is certainly possible for a single copy of code to be executed simultaneously by multiple physical processors. On Linux, that is how the operating system works. Much of the kernel supports concurrent multiprocessing these days (there isn't a giant global lock the way there used to be in Unix/Linux), and it would be silly to have multiple copies of the object code for the operating system, so the operating system's code is mapped into the address space of all of the processes, and all of the physical processors. In any case, this particular issue is probably irrelevant to the larger point, that is it is intended that calls from an active partition to a passive partition are permitted without having any particular limitations and without "marshaling," presuming the partitions are configured such that they are "connected," however that is defined. **************************************************************** From: Tucker Taft Sent: Tuesday, November 19, 2019 3:53 PM Another piece of the model we should mention is that partitions can come and go during the execution of an overall distributed program. In particular, a passive partition might outlive some of the active partitions, and represent a kind of persistent state. So partitions are expected to elaborate separately, and be loaded separately, and retain their data independently. When an active partition first connects to a passive partition, there should be no expectation that the data of the passive partition is still in some initial state. Some other active partition might have performed various operations on the data of the passive partition before the new active partition even started executing. This is described in paragraphs 13 and 14 of E.1: "In an implementation, the partitions of a distributed program need not be loaded and elaborated all at the same time; they may be loaded and elaborated one at a time over an extended period of time. An implementation may provide facilities to abort and reload a partition during the execution of a distributed program. An implementation may allow the state of some of the partitions of a distributed program to persist while other partitions of the program terminate and are later reinvoked." **************************************************************** From: Randy Brukardt Sent: Tuesday, November 19, 2019 4:04 PM > Our conclusions are probably the same but I don't understand your last > paragraph. You are saying that address mapping implies that it would > be accessible only from a single processor. I don't understand that. > When physical memory is shared between physical processors, it is > certainly possible for a single copy of code to be executed > simultaneously by multiple physical processors. ... Sure, but when physical memory is shared, you don't need to use Annex E. Especially with Ada 202x parallelism, you can just run the entire program and let the runtime do any distribution to multiple processors via lightweight threads (mapped explicitly or implicitly) and/or Ada tasks. The Annex E stuff is just additional complications and restrictions without much benefit in such a case. You need Annex E when the memory isn't shared, as in such cases it's impractical to map a single Ada program to multiple processors. Forcing shared passive partitions to be usable only on a single shared memory machine would take away much of the utility. Of course, you can use Annex E on a shared memory machine if you want, but that's not the primary purpose, and I wouldn't expect implementations to worry about that case. (I realize that some customer with heavy moneybags could change this, but that applies to any implementation technique.) An implementation like the AdaCore one only necessarily requires that the data file containing the state of the shared passive package be available somewhere on the network. That's much less restrictive than requiring the use of shared memory. Ergo, the primary implementation has to be shared (network-accessible) data/duplicated code. One can imagine a secondary implementation using shared memory for the rare cases where that would help, but it can't be the only implementation. **************************************************************** From: Jean-Pierre Rosen Sent: Wednesday, November 20, 2019 11:54 AM > An implementation like the AdaCore one only necessarily requires that > the data file containing the state of the shared passive package be > available somewhere on the network. That's much less restrictive than > requiring the use of shared memory. Ergo, the primary implementation > has to be shared > (network-accessible) data/duplicated code. One can imagine a secondary > implementation using shared memory for the rare cases where that would > help, but it can't be the only implementation. You don't require shared memory, but the intent of a passive partition is to model shared memory between different active partitions. Once again, think of it as a VME bus, with two processor boards and a memory board. The processors can communicate through the shared memory (and avoid race conditions, since protected types -without entries- are allowed in shared passive units). **************************************************************** From: Jeff Cousins Sent: Wednesday, November 20, 2019 12:06 PM > Once again, think of it as a VME bus, with two processor boards and a memory > board. The processors can communicate through the shared memory I've certainly implemented systems like that, and ones where the processor boards have dual port memory accessible from both the processor and the VME bus (the effect is the same. **************************************************************** From: Simon Wright Sent: Wednesday, November 20, 2019 12:15 PM > The processors can communicate through the shared memory (and > avoid race conditions, since protected types -without entries- are > allowed in shared passive units). May be allowed, but may be tricky to implement? And possibly quite board-dependent, which could lead to commercial difficulty for implementers. **************************************************************** From: Tucker Taft Sent: Wednesday, November 20, 2019 2:01 PM > You need Annex E when the memory isn't shared, as in such cases it's > impractical to map a single Ada program to multiple processors. > Forcing shared passive partitions to be usable only on a single shared > memory machine would take away much of the utility. Of course, you can > use Annex E on a shared memory machine if you want, but that's not the > primary purpose, and I wouldn't expect implementations to worry about > that case. (I realize that some customer with heavy moneybags could > change this, but that applies to any implementation technique.) I didn't want to impose any particular restriction about the location of the code. I was just merely saying that there is nothing precluding putting code on a shared memory card. But it would almost always be more efficient to duplicate the code. The key point is that each partition can be loaded and elaborated separately. How you manage to connect up the code is implementation dependent. Some sort of a "DLL" would be the most likely approach, I would think, and the O/S might choose to share a single "load" of a DLL into physical memory, or it might load a separate copy for each active partition making calls into it. All of this is outside the model. **************************************************************** From: Tucker Taft Sent: Wednesday, November 20, 2019 4:05 PM > May be allowed, but may be tricky to implement? And possibly quite > board-dependent, which could lead to commercial difficulty for implementers. Read/modify/write is fundamental to doing synchronization using shared memory. If that is not supported by the board, you are right that that would probably require a more complex implementation using, presumably, a message-based locking protocol. This doesn't seem unique to Ada. **************************************************************** From: Randy Brukardt Sent: Wednesday, November 20, 2019 5:43 PM >>The processors can communicate through the shared memory (and avoid >>race conditions, since protected types -without entries- are allowed >>in shared passive units). >May be allowed, but may be tricky to implement? And possibly quite >board-dependent, which could lead to commercial difficulty for implementers. Exactly. I could see implementing that for a particular customer, but general commercial products are likely to be centered around network communication rather than shared memory. And even if the data is in shared memory, it's still likely that you would want to duplicate the code (remote shared memory is likely to be slower than local memory). In addition, duplicated code can be inlined and otherwise optimized for the particular use; you can't do that for shared code (lest it not be shared anymore). So I still think code duplication is the most reasonable approach in most scenarios. ****************************************************************