Re: [PATCH v3 08/33] drm/ttm: Bound page (re)allocation per defragmentation move

From: Matthew Brost

Date: Thu Nov 19 2026 - 21:14:42 EST


On Mon, Nov 09, 2026 at 06:44:06PM -0800, Niranjana Vishwanathapura wrote:
> On Fri, Jul 10, 2026 at 07:55:54PM -0700, Matthew Brost wrote:
> > A defragmentation move reallocates an object's backing at the device's
> > beneficial order and harvests whatever the allocator cannot upgrade from
> > the old tt. For a very large object this can reallocate (and have the
> > driver re-copy) the entire backing in one move, which is expensive and
> > can stall concurrent work for an unbounded stretch.
>
> Assisted-by: LLM
>
> NIT...So, a time budge would also be helpful here along with the size
> budge?

That could be option but a size budget seems like reasonable place to start.

>
> >
> > Add an optional per-move byte budget so a large object can be upgraded in
> > slices across successive moves. Plumb a new
> > ttm_operation_ctx::defrag_bytes_remaining through to the pool allocator:
> > only pages actually (re)allocated at the beneficial order are charged
> > against it; pages harvested (borrowed) from defrag_old_tt are free and do
> > not count. Once the budget is exhausted the allocator stops upgrading and
> > completes the new tt by harvesting the remainder of the old tt as-is,
> > marking the tt sub-optimal (TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED) so the
> > driver re-queues it and a later move resumes the upgrade. The slices
> > already upgraded come back as free already-beneficial harvests on the
> > next pass, so each move spends budget only on new forward progress.
> >
> > The allocator decrements the budget as it allocates and writes the unused
> > remainder back on success, letting the caller compute how many bytes were
> > actually spent. A value of 0 means unlimited (upgrade the whole object in
> > one move), preserving the previous behaviour. This is only consulted on a
> > defragmentation move, i.e. when defrag_old_tt is set.
> >
> > Cc: Carlos Santa <carlos.santa@xxxxxxxxx>
> > Cc: Ryan Neph <ryanneph@xxxxxxxxxx>
> > Cc: Christian Koenig <christian.koenig@xxxxxxx>
> > Cc: Huang Rui <ray.huang@xxxxxxx>
> > Cc: Matthew Auld <matthew.auld@xxxxxxxxx>
> > Cc: Maarten Lankhorst <maarten.lankhorst@xxxxxxxxxxxxxxx>
> > Cc: Maxime Ripard <mripard@xxxxxxxxxx>
> > Cc: Thomas Zimmermann <tzimmermann@xxxxxxx>
> > Cc: David Airlie <airlied@xxxxxxxxx>
> > Cc: Simona Vetter <simona@xxxxxxxx>
> > Cc: dri-devel@xxxxxxxxxxxxxxxxxxxxx
> > Cc: linux-kernel@xxxxxxxxxxxxxxx
> > Cc: Thomas Hellström <thomas.hellstrom@xxxxxxxxxxxxxxx>
> > Assisted-by: GitHub_Copilot:claude-opus-4.8
> > Signed-off-by: Matthew Brost <matthew.brost@xxxxxxxxx>
> > ---
> > drivers/gpu/drm/ttm/ttm_pool.c | 101 ++++++++++++++++++++++++++++++++-
> > include/drm/ttm/ttm_bo.h | 14 +++++
> > 2 files changed, 112 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c
> > index 52f5e64f05dd..1673e81b8df8 100644
> > --- a/drivers/gpu/drm/ttm/ttm_pool.c
> > +++ b/drivers/gpu/drm/ttm/ttm_pool.c
> > @@ -77,6 +77,11 @@ struct ttm_pool_dma {
> > * @dma_addr: Pointer to the next tt dma_address entry to populate if any.
> > * @remaining_pages: Remaining pages to populate.
> > * @tt_caching: The requested cpu-caching for the pages allocated.
> > + * @defrag_bytes_remaining: Byte budget for newly (re)allocated pages on a
> > + * defragmentation move, seeded from ttm_operation_ctx::defrag_bytes_remaining.
> > + * Pages harvested (borrowed) from the old tt are free and do not count; once
> > + * this drops to zero the allocator stops upgrading and harvests the remainder.
> > + * On completion it holds the unused budget. 0 means unlimited (no defrag cap).
> > */
> > struct ttm_pool_alloc_state {
> > struct page **pages;
> > @@ -84,6 +89,7 @@ struct ttm_pool_alloc_state {
> > dma_addr_t *dma_addr;
> > pgoff_t remaining_pages;
> > enum ttm_caching tt_caching;
> > + s64 defrag_bytes_remaining;
> > };
> >
> > /**
> > @@ -780,6 +786,7 @@ static void ttm_pool_alloc_state_init(const struct ttm_tt *tt,
> > alloc->dma_addr = tt->dma_address;
> > alloc->remaining_pages = tt->num_pages;
> > alloc->tt_caching = tt->caching;
> > + alloc->defrag_bytes_remaining = 0;
> > }
> >
> > /*
> > @@ -885,8 +892,12 @@ static int ttm_pool_harvest_page(struct ttm_tt *old_tt, unsigned int order,
> >
> > static int ttm_pool_harvest_remaining(struct ttm_pool *pool,
> > struct ttm_tt *old_tt, pgoff_t off,
> > - struct ttm_pool_alloc_state *alloc)
> > + struct ttm_pool_alloc_state *alloc,
> > + bool *suboptimal)
> > {
> > + unsigned int beneficial = ttm_pool_beneficial_order(pool);
> > + pgoff_t bnr = 1UL << beneficial;
> > +
> > /*
> > * @off is always an old-tt chunk head here, never a tail page, so
> > * reading the per-chunk order below is safe. This holds because:
> > @@ -906,8 +917,23 @@ static int ttm_pool_harvest_remaining(struct ttm_pool *pool,
> > struct page *p = old_tt->pages[off];
> > unsigned int order = ttm_pool_page_order(pool, p);
> > pgoff_t nr = 1UL << order;
> > + pgoff_t num_pages = off + alloc->remaining_pages;
>
> num_pages seems loop invariant. May be move it out of the loop as total_pages
> or something?
>

No, both 'off' and 'alloc->remaining_pages' change each iteration of the loop.

> > int r;
> >
> > + /*
> > + * Only report a sub-optimal backing when a chunk we cannot
> > + * upgrade (out of budget/prealloc) sits in a beneficial-order
> > + * aligned region that fully fits in the tt: that region could
> > + * have been upgraded with more budget. A chunk below the
> > + * beneficial order is expected for the unaligned tail (the last
> > + * partial beneficial-order region) and for already beneficial-
> > + * order harvests - neither leaves real defrag work, so they must
> > + * not flag the tt for re-queue.
> > + */
> > + if (beneficial && order < beneficial &&
> > + round_down(off, bnr) + bnr <= num_pages)
> > + *suboptimal = true;
> > +
> > r = ttm_pool_harvest_page(old_tt, order, off, alloc);
> > if (r)
> > return r;

e.g., below here (unseen) 'off' is incremented by 'nr'.

> > @@ -951,6 +977,7 @@ struct ttm_pool_alloc_iter {
> > struct ttm_tt *defrag_old_tt;
> > unsigned int beneficial_order;
> > gfp_t gfp_flags;
> > + bool defrag_capped;
> >
> > /* Mutated as the loop walks the orders. */
> > unsigned int order;
> > @@ -962,6 +989,44 @@ struct ttm_pool_alloc_iter {
> > struct page *p;
> > };
> >
> > +/*
> > + * Defrag move budget exhausted: the upgrade can make no further progress this
> > + * pass. Snapshot @defrag_capped is set only when a byte budget was in force at
> > + * entry.
> > + */
> > +static bool ttm_pool_iter_defrag_exhausted(const struct ttm_pool_alloc_iter *it)
> > +{
> > + if (!it->defrag_old_tt)
> > + return false;
> > +
> > + return it->defrag_capped && it->alloc->defrag_bytes_remaining <= 0;
> > +}
> > +
> > +/*
> > + * Complete the new tt by harvesting the rest of the old tt as-is (borrowing its
> > + * pages, including any still below beneficial order). Never fall back to in-lock
> > + * reclaim for the shortfall - that would reintroduce the stall we preallocated
> > + * to avoid. Only re-queue (flag) the tt when the harvested remainder is
> > + * genuinely sub-optimal; if every remaining old chunk was already at beneficial
> > + * order the move fully upgraded the object.
> > + */
> > +static int ttm_pool_iter_harvest_rest(struct ttm_pool_alloc_iter *it)
> > +{
> > + pgoff_t off = it->tt->num_pages - it->alloc->remaining_pages;
> > + bool suboptimal = false;
> > + int r;
> > +
> > + r = ttm_pool_harvest_remaining(it->pool, it->defrag_old_tt, off,
> > + it->alloc, &suboptimal);
> > + if (r)
> > + return r;
> > +
> > + if (suboptimal)
> > + it->tt->page_flags |= TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED;
> > +
> > + return 0;
> > +}
> > +
> > /*
> > * Defrag move: reuse an already beneficial-order chunk from the old tt at this
> > * offset rather than reallocating it. Harvest whatever order the old chunk has
> > @@ -1062,11 +1127,12 @@ static int ttm_pool_iter_lower_order(struct ttm_pool_alloc_iter *it)
> > if (it->alloc_count && it->defrag_old_tt) {
> > pgoff_t off = it->tt->num_pages -
> > it->alloc->remaining_pages;
> > + bool suboptimal = false;
> > int r;
> >
> > r = ttm_pool_harvest_remaining(it->pool,
> > it->defrag_old_tt, off,
> > - it->alloc);
> > + it->alloc, &suboptimal);
>
> As suboptimal is not required here, probably send a NULL here and have
> ttm_pool_harvest_remaining handle that NULL.
>

That is better will change.

> > if (r)
> > return r;
> >
> > @@ -1121,10 +1187,23 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
> > gfp_flags |= GFP_HIGHUSER;
> >
> > it.gfp_flags = gfp_flags;
> > + it.defrag_capped = it.defrag_old_tt && alloc->defrag_bytes_remaining > 0;
> >
> > for (it.order = ttm_pool_alloc_find_order(MAX_PAGE_ORDER, alloc);
> > alloc->remaining_pages;
> > it.order = ttm_pool_alloc_find_order(it.order, alloc)) {
> > + /*
> > + * Out of defrag budget: harvest the rest of the old tt as-is and
> > + * stop (the tt is re-queued if the remainder is still
> > + * sub-optimal).
> > + */
> > + if (ttm_pool_iter_defrag_exhausted(&it)) {
> > + r = ttm_pool_iter_harvest_rest(&it);
> > + if (r)
> > + goto error_free_all;
> > + break;
> > + }
> > +
> > /* Reuse an already beneficial-order chunk from the old tt. */
> > r = ttm_pool_iter_reuse_old(&it);
> > if (r < 0)
> > @@ -1155,6 +1234,15 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
> > goto error_free_all;
> > }
> >
> > + /*
> > + * Charge a newly (re)allocated chunk against the defrag move
> > + * budget. Harvested chunks borrow the old tt's pages for free
> > + * and never reach here.
> > + */
> > + if (it.defrag_capped)
> > + alloc->defrag_bytes_remaining -=
> > + (s64)(1UL << it.order) << PAGE_SHIFT;
> > +
> > it.alloc_count++;
> > }
> >
> > @@ -1204,14 +1292,21 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
> > struct ttm_operation_ctx *ctx)
> > {
> > struct ttm_pool_alloc_state alloc;
> > + int ret;
> >
> > if (WARN_ON(ttm_tt_is_backed_up(tt)))
> > return -EINVAL;
> >
> > tt->page_flags &= ~TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED;
> > ttm_pool_alloc_state_init(tt, &alloc);
> > + alloc.defrag_bytes_remaining = ctx->defrag_bytes_remaining;
> > +
> > + ret = __ttm_pool_alloc(pool, tt, ctx, &alloc, NULL);
> > +
> > + /* Report the unused defrag budget back to the caller. */
> > + ctx->defrag_bytes_remaining = alloc.defrag_bytes_remaining;
> >
> > - return __ttm_pool_alloc(pool, tt, ctx, &alloc, NULL);
> > + return ret;
> > }
> > EXPORT_SYMBOL(ttm_pool_alloc);
> >
> > diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h
> > index fcaba5fb8bf7..6124d2627b47 100644
> > --- a/include/drm/ttm/ttm_bo.h
> > +++ b/include/drm/ttm/ttm_bo.h
> > @@ -223,6 +223,20 @@ struct ttm_operation_ctx {
> > * NULL for non-defrag operations.
> > */
> > struct ttm_tt *defrag_old_tt;
> > + /**
> > + * @defrag_bytes_remaining: Byte budget for newly allocated pages during
> > + * a defragmentation move. Only pages that are actually (re)allocated at
> > + * the beneficial order count against it; pages harvested (borrowed) from
> > + * @defrag_old_tt are free and do not. When the budget is exhausted the
> > + * pool allocator stops upgrading and completes the new tt by harvesting
> > + * the remainder of @defrag_old_tt, marking the tt sub-optimal so the
> > + * driver can resume on a later pass. The allocator decrements this as it
> > + * allocates, so on return it holds the unused budget (which the caller
> > + * can use to compute how many bytes were spent). A value of 0 means
> > + * unlimited (upgrade the whole object). Only consulted when
> > + * @defrag_old_tt is set.
> > + */
> > + s64 defrag_bytes_remaining;
>
> If user specifies -ve value, currently it means unlimited. So, may be update the
> documentation here to say "A value of <= 0 means unlimited"? Or, better only accept
> > =0 value from the user ctx here.

How about I reject user input defrag_bytes_remaining < 0?

Matt

>
> Niranjana
>
> > /**
> > * @resv: Reservation object to be used together with
> > * @allow_res_evict.
> > --
> > 2.34.1
> >