PFNs busy - CMA memory stolen from percpu

From: Rafal
Date: Tue Oct 17 2017 - 10:27:45 EST


I have an arm64 board which uses cma allocator for VPU (NanoPi M3). It has 96MB of cma-reserved memory and sometimes cma allocation fails, despite the memory amount was set with some reserve. Moreover, higher memory reserve helps only to some extent. With 256MB of cma memory the system encounters many "PFNs busy" messages in kernel log and overall slowdown at video startup.

I have made some investigations. It looks that usual migratable allocations steal plenty of cma memory from per-cpu pool, for e.g. shared memory, despite there is a lot of "normal" (non-cma) memory which might be used for that. Most of the memory is given back when a new cma allocation is made, but some (rare) pages aren't reclaimed and result with the "PFNs busy" messages. Please imagine: even if one page per 100 is not reclaimed, it is a problem for contiguous HD video buffers allocation, about 2MB each.

I have made a test: I have added drain_all_pages() function call at end of free_contig_range() call. The "PFNs busy" messages disappeared completely and video playing is started more fluently. So, I'm wondering whether cma pages should be really mixed with usual migratable pages in percpu pool?

Rafal