Interprocess Communication in L4Re

There are several layers of interfaces and mechanisms for interprocess communication (IPC) in L4Re, alongside various fundamental concepts.

Messages

IPC operations define messages using a tag abstraction with the following properties:

Messages employ "virtual registers" found in the user-level thread control block (UTCB):

For items to be successfully sent, the receiver must declare the expected items in its buffer registers. So, a call from a thread that sends an item must be received by a thread declaring an expected item:

call_itemscallermr[0]itemtargetmr[0]br[0] = expected

If a thread, having made a call, expects items in return, it too must declare these items in the buffer registers:

expect_itemscallermr[0]br[0] = expecteditemtargetmr[0]

Failure to declare expected items will cause an "overflow" or "sent message cut" error. Thus, the necessary steps are as follows:

  1. Reserve a capability slot for any expected capability
  2. Declare the expected capability using the slot
  3. Perform a wait operation
  4. Test for any items providing a capability
  5. Obtain any capability
  6. Start again (by providing a new slot for any expected capability)

A limitation exists in combining words and items. Since sent items are not counted but sent words are, words must be placed in the message registers before any items. Received items are copied into corresponding positions in the receiver's message registers.

Unless the buffer descriptor register has been adjusted, expected items are declared in the buffer registers starting at register 0, with each item corresponding to the items as they are encountered after the words in the message registers.

words_and_itemscallermr[0]mr[1]worditemtargetmr[0] mr[1]br[0] = expected

The buffer descriptor register indicates the positions of items in the buffer registers as well as various flags. The Fiasco.OC kernel code has more details about the register structure:

The following layout applies to the buffer descriptor register in Fiasco.OC:

Bits

31..24

23..15

14..10

9..5

4..0

Flags

First capability item

First I/O buffer item

First memory buffer item

It would seem that a common value for the register is zero, indicating that items are present from position zero onwards.

Details of the tag abstraction (l4_msgtag_t) and associated functions can be found here:

Items and Flexpages

The flexpage is an abstraction used to describe memory, capabilities and other things. Items employ flexpage values along with additional details:

It appears possible to just specify the appropriate metadata in the "base". For example:

mr[0] = 0 | L4_ITEM_MAP | L4_ITEM_CONT;

Here, the superfluous 0 merely indicates that no base capability is specified.

Long IPC

According to L4 Microkernels: The Lessons from 20 Years of Research and Deployment, "long IPC" is no longer supported in modern L4 implementations including Fiasco.OC. This is presumably the form of IPC described in various educational materials where the kernel copies data from the sender directly into the recipient's address space. Alternatives to this include explicit sharing of memory between sender and recipient.

IPC Gates and Labels

When binding a thread to an existing IPC gate (using l4_rcv_ep_bind_thread) or doing so when creating the gate (l4_factory_create_gate), a label can be associated with the gate which can then be inspected upon receiving messages. This label allows the gate through which the message arrived to be identified.

Consequently, gates may be associated with objects or structures that can be used to handle incoming messages by providing the address of such an object or structure as the label. Since the label is defined by the recipient and preserved by the gate, invalid addresses cannot be presented as labels by message senders.

The lowest two bits of a label is reserved for rights flags. Thus, addresses need to be aligned to four-byte boundaries, which is already typical on various architectures.

l4_ipc

The l4_ipc functions, defined here:

These expose operations acting more directly with capabilities, message tags and the UTCB than other IPC interface layers. Only very limited support for updating the virtual registers is provided, this in the form of the l4_sndfpage_add and l4_sndfpage_add_u functions which add "items" for flexpage values to the message registers.

Flexpage definitions can be found here:

The principal function is the l4_ipc function itself, with various other functions employing it to perform specialised IPC operations.

ipc_stream

A "stream" abstraction is provided for IPC clients and servers for the modification of the virtual registers.

Server-Side IPC Framework

The server-side IPC framework provides abstractions for server-like objects interpreting and dispatching IPC messages, with registry objects provided to dispatch messages to servers based on message labels.

Useful things to note:

Kernel Activities

Message Items

Message items are handled in the Thread::transfer_msg_items method found here:

Each item is read from the sending UTCB:

According to the type of the item (Memory, Io or Obj, with the latter referring to general capabilities), the appropriate buffer register region is selected for iteration, such regions being defined by the buffer descriptor register:

Where buffer registers do not describe expected items that match the incoming items, errors are raised. Missing descriptions cause an "overflow" error.

Flexpages, being items, are defined here: