Re: RFC: Memory Tiering Kernel Interfaces (v2)

[Mika Penttilä]

On 25.5.2022 13.01, Aneesh Kumar K V wrote:
> On 5/25/22 2:33 PM, Ying Huang wrote:
> > On Tue, 2022-05-24 at 22:32 -0700, Wei Xu wrote:
> > > On Tue, May 24, 2022 at 1:24 AM Ying Huang <ying.huang@xxxxxxxxx> wrote:
> > > > On Tue, 2022-05-24 at 00:04 -0700, Wei Xu wrote:
> > > > > On Thu, May 19, 2022 at 8:06 PM Ying Huang <ying.huang@xxxxxxxxx> 
> > > > > wrote:
> > > > > >
>
> ...
>
> > OK.  Just to confirm.  Does this mean that we will have fixed device ID,
> > for example,
> >
> > GPU            memtier255
> > DRAM (with CPU)        memtier0
> > PMEM            memtier1
> >
> > When we add a new memtier, it can be memtier254, or memter2?  The rank
> > value will determine the real demotion order.
> >
> > I think you may need to send v3 to make sure everyone is at the same
> > page.
>
> What we have implemented which we will send as RFC shortly is below.
>
> cd /sys/dekvaneesh@ubuntu-guest:~$ cd /sys/devices/system/
> kvaneesh@ubuntu-guest:/sys/devices/system$ pwd
> /sys/devices/system
> kvaneesh@ubuntu-guest:/sys/devices/system$ ls
> clockevents  clocksource  container  cpu  edac  memory  memtier  mpic 
> node  power
> kvaneesh@ubuntu-guest:/sys/devices/system$ cd memtier/
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ pwd
> /sys/devices/system/memtier
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ ls
> default_rank  max_rank  memtier1  power  uevent
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat default_rank
> 1
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat max_rank
> 3
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cd memtier1/
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ ls
> nodelist  power  rank  subsystem  uevent
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat nodelist
> 0-3
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat rank
> 1
> kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cd 
> ../../node/node1/
> kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ cat memtier
> 1
> kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$
> root@ubuntu-guest:/sys/devices/system/node/node1# echo 0 > memtier
> root@ubuntu-guest:/sys/devices/system/node/node1# cat memtier
> 0
> root@ubuntu-guest:/sys/devices/system/node/node1# cd ../../memtier/
> root@ubuntu-guest:/sys/devices/system/memtier# ls
> default_rank  max_rank  memtier0  memtier1  power  uevent
> root@ubuntu-guest:/sys/devices/system/memtier# cd memtier0/
> root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat nodelist
> 1
> root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat rank
> 0
> root@ubuntu-guest:/sys/devices/system/memtier/memtier0# echo 4 > rank
> bash: rank: Permission denied
> root@ubuntu-guest:/sys/devices/system/memtier/memtier0#

Just to confirm, unlike today's demotion code, the demotion target 
allocation is planned to honor mempolicies?


--Mika