Re: [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind()

From: Dave Hansen
Date: Wed Dec 05 2018 - 12:27:21 EST


On 12/4/18 6:13 PM, Jerome Glisse wrote:
> On Tue, Dec 04, 2018 at 05:06:49PM -0800, Dave Hansen wrote:
>> OK, but there are 1024*1024 matrix cells on a systems with 1024
>> proximity domains (ACPI term for NUMA node). So it sounds like you are
>> proposing a million-directory approach.
>
> No, pseudo code:
> struct list links;
>
> for (unsigned r = 0; r < nrows; r++) {
> for (unsigned c = 0; c < ncolumns; c++) {
> if (!link_find(links, hmat[r][c].bandwidth,
> hmat[r][c].latency)) {
> link = link_new(hmat[r][c].bandwidth,
> hmat[r][c].latency);
> // add initiator and target correspond to that row
> // and columns to this new link
> list_add(&link, links);
> }
> }
> }
>
> So all cells that have same property are under the same link.

OK, so the "link" here is like a cable. It's like saying, "we have a
network and everything is connected with an ethernet cable that can do
1gbit/sec".

But, what actually connects an initiator to a target? I assume we still
need to know which link is used for each target/initiator pair. Where
is that enumerated?

I think this just means we need a million symlinks to a "link" instead
of a million link directories. Still not great.

> Note that userspace can parse all this once during its initialization
> and create pools of target to use.

It sounds like you're agreeing that there is too much data in this
interface for applications to _regularly_ parse it. We need some
central thing that parses it all and caches the results.