Re: [PATCH] Documentation: dev-tools: Add Testing Overview

[Brendan Higgins]

On Mon, Apr 12, 2021 at 3:43 AM Marco Elver <elver@xxxxxxxxxx> wrote:
>
> On Sat, 10 Apr 2021 at 13:53, Daniel Latypov <dlatypov@xxxxxxxxxx> wrote:
> > On Sat, Apr 10, 2021 at 12:05 AM David Gow <davidgow@xxxxxxxxxx> wrote:
> [...]
> > > +
> > > +
> > > +Sanitizers
> > > +==========
> > > +
>
> The "sanitizers" have originally been a group of tools that relied on
> compiler instrumentation to perform various dynamic analysis
> (initially ASan, TSan, MSan for user space). The term "sanitizer" has
> since been broadened to include a few non-compiler based tools such as
> GWP-ASan in user space, of which KFENCE is its kernel cousin but it
> doesn't have "sanitizer" in its name (because we felt GWP-KASAN was
> pushing it with the acronyms ;-)). Also, these days we have HW_TAGS
> based KASAN, which doesn't rely on compiler instrumentation but
> instead on MTE in Arm64.
>
> Things like kmemleak have never really been called a sanitizer, but
> they _are_ dynamic analysis tools.
>
> So to avoid confusion, in particular avoid establishing "sanitizers"
> to be synonymous with "dynamic analysis" ("all sanitizers are dynamic
> analysis tools, but not all dynamic analysis tools are sanitizers"),
> the section here should not be called "Sanitizers" but "Dynamic
> Analysis Tools". We could have a subsection "Sanitizers", but I think
> it's not necessary.
>
> > > +The kernel also supports a number of sanitizers, which attempt to detect
> > > +classes of issues when the occur in a running kernel. These typically
> >
> > *they occur
> >
> > > +look for undefined behaviour of some kind, such as invalid memory accesses,
> > > +concurrency issues such as data races, or other undefined behaviour like
> > > +integer overflows.
> > > +
> > > +* :doc:`kmemleak` (Kmemleak) detects possible memory leaks.
> > > +* :doc:`kasan` detects invalid memory accesses such as out-of-bounds and
> > > +  use-after-free errors.
> > > +* :doc:`ubsan` detects behaviour that is undefined by the C standard, like
> > > +  integer overflows.
> > > +* :doc:`kcsan` detects data races.
> > > +* :doc:`kfence` is a low-overhead detector of memory issues, which is much
> > > +  faster than KASAN and can be used in production.
> >
> > Hmm, it lives elsewhere, but would also calling out lockdep here be useful?
> > I've also not heard anyone call it a sanitizer before, but it fits the
> > definition you've given.
> >
> > Now that I think about it, I've never looked for documentation on it,
> > is this the best page?
> > https://www.kernel.org/doc/html/latest/locking/lockdep-design.html
>
> Not a "sanitizer" but our sanitizers are all dynamic analysis tools,
> and lockdep is also a dynamic analysis tool.
>
> If we want to be pedantic, the kernel has numerous options to add
> "instrumentation" (compiler based or explicit) that will detect some
> kind of error at runtime. Most of them live in lib/Kconfig.debug. I
> think mentioning something like that is in scope of this document, but
> we certainly can't mention all debug tools the kernel has to offer.
> Mentioning the big ones like above and then referring to
> lib/Kconfig.debug is probably fine.
>
> Dmitry recently gave an excellent talk on some of this:
> https://www.youtube.com/watch?v=ufcyOkgFZ2Q

Good point Marco, and we (KUnit - myself, Daniel, and David) gave a
talk on KUnit at LF. Also, I think Shuah is/has given one (soon)?
Might be a good idea to link those here?