[RFC PATCH 0/7] kvm: arm64: Support stage2 hardware DBM

From: Keqian Zhu
Date: Mon May 25 2020 - 07:25:10 EST


This patch series add support for stage2 hardware DBM, and it is only
used for dirty log for now.

It works well under some migration test cases, including VM with 4K
pages or 2M THP. I checked the SHA256 hash digest of all memory and
they keep same for source VM and destination VM, which means no dirty
pages is missed under hardware DBM.

However, there are some known issues not solved.

1. Some mechanisms that rely on "write permission fault" become invalid,
such as kvm_set_pfn_dirty and "mmap page sharing".

kvm_set_pfn_dirty is called in user_mem_abort when guest issues write
fault. This guarantees physical page will not be dropped directly when
host kernel recycle memory. After using hardware dirty management, we
have no chance to call kvm_set_pfn_dirty.

For "mmap page sharing" mechanism, host kernel will allocate a new
physical page when guest writes a page that is shared with other page
table entries. After using hardware dirty management, we have no chance
to do this too.

I need to do some survey on how stage1 hardware DBM solve these problems.
It helps if anyone can figure it out.

2. Page Table Modification Races: Though I have found and solved some data
races when kernel changes page table entries, I still doubt that there
are data races I am not aware of. It's great if anyone can figure them out.

3. Performance: Under Kunpeng 920 platform, for every 64GB memory, KVM
consumes about 40ms to traverse all PTEs to collect dirty log. It will
cause unbearable downtime for migration if memory size is too big. I will
try to solve this problem in Patch v1.

Keqian Zhu (7):
KVM: arm64: Add some basic functions for hw DBM
KVM: arm64: Set DBM bit of PTEs if hw DBM enabled
KVM: arm64: Traverse page table entries when sync dirty log
KVM: arm64: Steply write protect page table by mask bit
kvm: arm64: Modify stage2 young mechanism to support hw DBM
kvm: arm64: Save stage2 PTE dirty info if it is coverred
KVM: arm64: Enable stage2 hardware DBM

arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_mmu.h | 44 +++++-
arch/arm64/include/asm/pgtable-prot.h | 1 +
arch/arm64/include/asm/sysreg.h | 2 +
arch/arm64/kvm/reset.c | 9 +-
virt/kvm/arm/arm.c | 6 +-
virt/kvm/arm/mmu.c | 202 ++++++++++++++++++++++++--
7 files changed, 246 insertions(+), 19 deletions(-)

--
2.19.1