Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

[Ingo Molnar]

* Sam Ravnborg <sam@xxxxxxxxxxxx> wrote:

> > > 
> > > So just to repeat - it is an error prone design to let users
> > > of the kernel uapi maintain their own copies of the kernel
> > > uapi header. It is the job of the kernel.
> > 
> > But "random copies" is not what perf does. Tell me, how is the perf mechanism of 
> > using the headers "error-prone"? It's a delayed COW mechanism - COW is not an 
> > error-prone concept in any way ...
> 
> The whole concept that user space have the burden to maintain a set of headers 
> describing the uapi provided by the kernel is the point of discussion.

That's your characterisation and I think it's misleading: they don't have to 
"maintain a set of headers", they can, if they'd like to:

   "maintain a _copy_ of a small number of headers that the typical user-space 
    tooling project is interested in"

which is different from how you framed it.

> The randomness come into play when a user space developer are faced with the 
> challenge that the programm require access to something described by the kernel 
> uapi and then have to hunt for a header that describes said uapi.
>
> In this thread we have covered one rational reason to push thus
> burden to user space - to give the kernel the freedom to repair
> past stupidity (being that in naming or some other sort).

There's no real "burden" for heaven's sake: it's having to execute a 'cp' every 
now and then and check whether the result still builds (it will build just fine in 
the overwhelming majority of cases).

> So lets turn around the arguments - and from a user space
> perspective what is the benefit of maintaining a set of headers
> describing the kernel uapi?

Firsty, the headers are not maintained by the user-space project, 99.999% of the 
maintenance is done by the kernel developers.

> Obviously this allows user space to name thing exactly the way they like, and 
> allows user space to put all sorts of strange things in the header files 
> describing the kernel uapi.

Huh? Again, my suggestion is to to _copy_ the kernel header the tooling project is 
interested in as-is, and this is exactly what perf does. We don't change the UAPI 
headers, we copy them verbatim and check during the build that the copy is exact.

> Are there other benefits that is missed which makes the
> concept of letting user space maintain header files describing
> the kernel uapi a good idea that is missed?

Yes, you missed a lot of the benefits.

Firstly, the user-space tooling project that relies on some UAPI header with Linux 
kernel ABI details in it, if it so wishes, maintains a _copy_ of the affected 
headers, which is vastly less work and 'burden' than 'maintaining headers'.

Secondly, doing this copy has numerous advantages over using the UAPI headers from 
the distro installation:

 - Additions to the ABI are monitored by the tooling project. Say there's 
   external tooling that relies on the perf system call, and uses a copy of 
   perf_events.h. It can see new ABI additions in new versions of the file, and 
   can add support for them. If on the other hand the file is just shared with 
   whatever distro kernel headers there are on the system, there's no information 
   about ABI progress.

   Advantage: changes to the ABI can be seen in the tool's Git repository,
              the "git log" of the header shows how the versions got updated and 
              how the tool adopted to new ABI additions.

 - The tooling project is more self-contained: it can build just fine in more 
   limited environments that don't have the kernel headers installed.

   Advantage: more robust tool build process, fewer dependencies.

 - So the tooling project can copy the kernel header _before_ the updated kernel
   header trickles through to distros. This has the advantage of being able to add
   tooling support for new ABI additions before it all goes into the 'official'
   UAPI headers a distro includes, and speeds up ABI propagation.

   Advantage: faster ABI propagation to tooling.

 - In fact this model allows the tooling project to host new kernel ABI additions 
   in its development process, so sometimes it has the new UAPI header _before_ 
   the upstream kernel has it. (Obviously the upstream kernel has to pull it for 
   it to become the official UAPI header, so this only works for subsystems where
   there's good communication between the tooling developers and the kernel 
   developers.)

   This is what tools/perf does quite often.

   Advantage: better development model where tooling can develop new kernel 
              features hand in hand with kernel developers.

I.e. most of the advantages of the copied headers are for the _tooling project_, 
not for the kernel.

Your one sided characterization of the copying method as a 'burden' ignores all 
that.

Thanks,

	Ingo