The following is our understanding of the proposed User Interrupt.
We have been exploring how user-level interrupts (UIs) can be used to
improve performance and programmability in several different areas:
e.g., parallel programming, memory management, I/O, and floating-point
libraries.
# Current Use Cases
The Current RFC is focused on sending an interrupt from one user-space
thread (UST) to another user-space thread (UST2UST). These threads
could be in different processes, as long as the sender has access to
the receiver's User Interrupt File Descriptor (uifd). Based on our
understanding, UIs are currently targeted as a low overhead
alternative for the current IPC mechanisms.
# Preparing for future use cases
> If someone could point out an example for Kernel to
user-space thread (K2UST) UI, we would appreciate it.
In our work, we have also been exploring precise UIs from the
currently running thread. We call these CPU to UST (CPU2UST) UIs.
For example, a SIGSEGV generated by writing to a read-only page, a
SIGFPE generated by dividing a number by zero.
- QUESTION: Is there is a rough draft/plan that we can refer to that describes the
current thinking on these three cases.
- QUESTION: Are there use cases for K2UST, or is K2UST the same as CPU2UST?
# Basic Understanding
First, we would like to make sure that our understanding of the terminology and the data structures is correct.
- User Interrupt Vector (UIV): The identity of the user interrupt.
- User Interrupt Target Table (UITT):
This allows the sender to locate the "address" of the receiver through the uifd.
Below outlines our understanding of the current API for UIs.
- Each thread that can receive UIs has exactly one handler
registered with `uintr_register_handler` (a syscall).
- Each thread that registers a handler calls `uintr_create_fd` for
every user-level interrupt vector (UIV) that they expect to receive.
- The only information delivered to the handler is the UIV.
- There are 64 UIVs that can be used per thread.
- A thread that wants to send a UI must register the receiver's uifd with `uintr_register_sender` (a syscall).
This returns an index the sender uses to locate the receiver.
- `_senduipi(index)` sends a user interrupt to a particular destination.
The sender's UITT and index determine the destination.
- A thread uses `_stui` (and `_clui`) to enable (and disable) the reception of UIs.
- As for now, there is no mechanism to mask a particular UIV.
- A UI is delivered to the receiver immediately only if it is currently running.
- If a thread executes the `uintr_wait()`, it will be scheduled only after receiving a UI.
There is no guarantee on the delay between the processor receiving the UI and when the thread is scheduled.
- If a thread is the target of a UI and another thread is running, or the target thread is blocked in the kernel,
then the target thread will handle the UI when it is next scheduled.
- Ordinary interrupts (interrupt delivered with CPL=0) have a higher priority over user interrupts.
- UI handler only saves general-purpose registers (e.g., do not save floating-point registers).
- User Interrupts with higher UIV are given a higher priority than those with smaller UIV.
## Private UITT
The Current RFC focuses on a private UITT where each thread has its own
UITT. Thus, different threads executing `_senduipi(index1)` with the
same `index1` may cause different receiver threads to be interrupted.
In many cases, the receiver of an interrupt needs to know which thread
sent the interrupt. If we understand the proposal correctly, there are
only 64 user-level interrupt vectors (UIVs), and the UIV is the only
information transmitted to the receiver. The UIV itself allows the
receiver to distinguish different senders through careful management
of the receiver's UIV.
- QUESTION: Given the code below where the same UIV is registered twice:
```c
uintr_fd1 = uintr_create_fd(vector1, flags)
uintr_fd2 = uintr_create_fd(vector1, flags)
```
Would `uintr_fd1` be the same as `uintr_fd2`, or would it be registered with a different index in the UITT table?
- QUESTION: If it is registered in a different index, would the
receiver be able to distinguish the sender if `uintr_fd1` and
`uintr_fd2` are used from two different threads?
- QUESTION: What is the intended future use of the `flags` argument?
## Shared UITT
In the case of the shared UITT model, all the threads share the same
UITT and thus, if two different threads execute `_senduipi(index)`
with the same index, they would both cause an interrupt in the
same destination/receiver.
- QUESTION: Since both threads use the same entry (same
destination/receiver), does this mean that the receiver will not be
able to distinguish the sender of the interrupt?
# Multi-threaded parallel programming example
One of the uses for UIs that we have been exploring is combining the
message-passing and shared memory models for parallel programming. In
our approach, message-passing is used for synchronization and shared
memory for data sharing. The message passing part of the programming
pattern is based loosely on Active Messages (See ISCA92), where a
particular thread can turn off/on interrupts to ignore incoming
messages so they can execute critical sections without having to
notify any other threads in the system.
- QUESTION: Is there any data on the performance impact of `_stui` and `_clui`?