Re: [PATCH] perf_events: improve DS/BTS/PEBS buffer allocation

[Stephane Eranian]

On Mon, Sep 13, 2010 at 5:09 PM, Frederic Weisbecker <fweisbec@xxxxxxxxx> wrote:
> On Mon, Sep 13, 2010 at 04:55:01PM +0200, Stephane Eranian wrote:
>> The DS, BTS, and PEBS memory regions were allocated using kzalloc(), i.e.,
>> requesting contiguous physical memory. There is no such restriction on
>> DS, PEBS and BTS buffers. Using kzalloc() could lead to error in case
>> no contiguous physical memory is available. BTS is requesting 64KB,
>> thus it can cause issues. PEBS is currently only requesting one page.
>> Both PEBS and BTS are static buffers allocated for each CPU at the
>> first user. When the last user exists, the buffers are released.
>>
>> All buffers are only accessed on the CPU they are attached to.
>> kzalloc() does not take into account NUMA, thus all allocations
>> are taking place on the NUMA node where the perf_event_open() is
>> made.
>>
>> This patch switches allocation to vmalloc_node() to use non-contiguous
>> physical memory and to allocate on the NUMA node corresponding to each
>> CPU. We switched DS and PEBS although they do not cause problems today,
>> to, at least, make the allocation on the correct NUMA node. In the future,
>> the PEBS buffer size may increase. DS may also grow bigger than a page.
>> This patch eliminates the memory allocation imbalance.
>>
>> vmalloc_node() returns page-aligned addresses which do conform with the
>> restriction on PEBS buffer as documented by Intel in Vol3a section 16.9.4.2.
>>
>> Signed-off-by: Stephane Eranian <eranian@xxxxxxxxxx>
>> --
>
>
> For now I think you can not do this. vmalloc'ed memory can't be safely
> accessed from NMIs in x86 because that might fault. And faults from NMIs
> are not supported. They cause very bad things: return from fault calls
> iret which reenables NMI, so NMI can nest but in the meantime there is
> only one NMI stack, so that gets quickly messed up.
>
What kind of faults are you talking about here? TLB faults?

But I don't want contiguous memory. This puts unnecessary pressure on
the memory subsystem. I have seen failures on my system because it
could not find 64KB of contiguous physical, but there was clearly more
than 64kb of physical memory available. And I want NUMA local allocations
as well.


>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/