Re: [RFC PATCH v2 11/17] sched: Basic tracking of matching tasks

From: Subhra Mazumdar
Date: Wed May 08 2019 - 22:17:16 EST



On 5/8/19 6:38 PM, Aubrey Li wrote:
On Thu, May 9, 2019 at 8:29 AM Subhra Mazumdar
<subhra.mazumdar@xxxxxxxxxx> wrote:

On 5/8/19 5:01 PM, Aubrey Li wrote:
On Thu, May 9, 2019 at 2:41 AM Subhra Mazumdar
<subhra.mazumdar@xxxxxxxxxx> wrote:
On 5/8/19 11:19 AM, Subhra Mazumdar wrote:
On 5/8/19 8:49 AM, Aubrey Li wrote:
Pawan ran an experiment setting up 2 VMs, with one VM doing a
parallel kernel build and one VM doing sysbench,
limiting both VMs to run on 16 cpu threads (8 physical cores), with
8 vcpu for each VM.
Making the fix did improve kernel build time by 7%.
I'm gonna agree with the patch below, but just wonder if the testing
result is consistent,
as I didn't see any improvement in my testing environment.

IIUC, from the code behavior, especially for 2 VMs case(only 2
different cookies), the
per-rq rb tree unlikely has nodes with different cookies, that is, all
the nodes on this
tree should have the same cookie, so:
- if the parameter cookie is equal to the rb tree cookie, we meet a
match and go the
third branch
- else, no matter we go left or right, we can't find a match, and
we'll return idle thread
finally.

Please correct me if I was wrong.

Thanks,
-Aubrey
This is searching in the per core rb tree (rq->core_tree) which can have
2 different cookies. But having said that, even I didn't see any
improvement with the patch for my DB test case. But logically it is
correct.

Ah, my bad. It is per rq. But still can have 2 different cookies. Not sure
why you think it is unlikely?
Yeah, I meant 2 different cookies on the system, but unlikely 2
different cookies
on one same rq.

If I read the source correctly, for the sched_core_balance path, when try to
steal cookie from another CPU, sched_core_find() uses dst's cookie to search
if there is a cookie match in src's rq, and sched_core_find() returns idle or
matched task, and later put this matched task onto dst's rq (activate_task() in
sched_core_find()). At this moment, the nodes on the rq's rb tree should have
same cookies.

Thanks,
-Aubrey
Yes, but sched_core_find is also called from pick_task to find a local
matching task.
Can a local searching introduce a different cookies? Where is it from?
No. I meant the local search uses the same binary search of sched_core_find
so it has to be correct.

The enqueue side logic of the scheduler is unchanged with
core scheduling,
But only the task with cookies is placed onto this rb tree?

so it is possible tasks with different cookies are
enqueued on the same rq. So while searching for a matching task locally
doing it correctly should matter.
May I know how exactly?
select_task_rq_* seems to be unchanged. So the search logic to find a cpu
to enqueue when a task becomes runnable is same as before and doesn't do
any kind of cookie matching.

Thanks,
-Aubrey