Re: Rseq registration: Google tcmalloc vs glibc

[Mathieu Desnoyers]

----- On Feb 21, 2020, at 10:49 AM, Joel Fernandes, Google joel@xxxxxxxxxxxxxxxxx wrote:

[...]
>> 
>> 3) Use the  __rseq_abi TLS cpu_id field to know whether Rseq has been
>> registered.
>> 
>> - Current protocol in the most recent glibc integration patch set.
>> - Not supported yet by Linux kernel rseq selftests,
>> - Not supported yet by tcmalloc,
>> 
>> Use the per-thread state to figure out whether each thread need to register
>> Rseq individually.
>> 
>> Works for integration between a library which exists for the entire lifetime
>> of the executable (e.g. glibc) and other libraries. However, it does not
>> allow a set of libraries which are dlopen'd/dlclose'd to co-exist without
>> having a library like glibc handling the registration present.
> 
> Mathieu, could you share more details about why during dlopen/close
> libraries we cannot use the same __rseq_abi TLS to detect that rseq was
> registered?

Sure,

A library which is only loaded and never closed during the execution of the
program can let the kernel implicitly unregister rseq at thread exit. For
the dlopen/dlclose use-case, we need to be able to explicitly unregister
each thread's __rseq_abi which sit in a library which is going to be
dlclose'd. 

The issue is that __rseq_abi.cpu_id does not track any reference counting
of rseq user libraries, which becomes an issue if we have many of those
libraries around with different life-time.

As an example scenario, let's suppose we have a single-threaded application
which does the following:

main()
  dlopen(liba)
    -> liba's constructor observes uninitialized __rseq_abi.cpu_id, thus
       performs rseq registration
  dlopen(libb)
    -> libb's constructor observes that rseq is already registered.

  dlclose(libb)
    -> libb's destructor unregisters rseq.

  -> at this point, liba is still loaded, and would still expect rseq to
     be registered. But unfortunately rseq has been unregistered by libb.

  dlclose(liba)
    -> rseq is already unregistered, which is unexpected.

The TLS __rseq_refcount solves this by tracking the number of users of
rseq for the thread, so rseq is only unregistered when the very last user
decrements the reference count.

As soon as there is at least one library taking care of registering rseq
for the entire thread's duration (e.g. glibc), and that this library
guarantees to never be dlclose'd, the __rseq_refcount becomes unneeded.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com