Re: [PATCH v7 053/102] KVM: TDX: don't request KVM_REQ_APIC_PAGE_RELOAD

From: Yuan Yao
Date: Mon Jul 11 2022 - 23:47:52 EST


On Mon, Jun 27, 2022 at 02:53:45PM -0700, isaku.yamahata@xxxxxxxxx wrote:
> From: Isaku Yamahata <isaku.yamahata@xxxxxxxxx>
>
> TDX doesn't need APIC page depending on vapic and its callback is
> WARN_ON_ONCE(is_tdx). To avoid unnecessary overhead and WARN_ON_ONCE(),
> skip requesting KVM_REQ_APIC_PAGE_RELOAD when TD.
>
> ------------[ cut here ]------------
> WARNING: CPU: 134 PID: 42205 at arch/x86/kvm/vmx/main.c:696 vt_set_apic_access_page_addr+0x3c/0x50 [kvm_intel]
> Modules linked in: squashfs nls_iso8859_1 nls_cp437 vhost_vsock vhost vhost_iotlb tdx_debug kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd i2c_i801 i2c_smbus i2c_ismt
> CPU: 134 PID: 42205 Comm: tdx_vm_tests Tainted: G W 5.17.0-rc8 #165 4baba67c36c7c1001d782c47f2964b779a5659c7
> Hardware name: Intel Corporation EAGLESTREAM/EAGLESTREAM, BIOS EGSDCRB1.SYS.0066.D24.2110072326 10/07/2021
> RIP: 0010:vt_set_apic_access_page_addr+0x3c/0x50 [kvm_intel]
> Code: e7 d5 49 8b 1c 24 48 8d bb 78 15 00 00 e8 4c 78 e7 d5 48 83 bb 78 15 00 00 01 74 0d 4c 89 e7 e8 7a 9b fd ff 5b 41 5c 5d c3 90 <0f 0b 90 5b 41 5c 5d c3 66 66 2e 0f 1f 84 00 00 00 00 00 90 0f 1f
> RSP: 0018:ffa0000027477b68 EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffa00000572d9000 RCX: ffffffffde6864d4
> RDX: dffffc0000000000 RSI: 0000000000000008 RDI: ffa00000572da578
> RBP: ffa0000027477b78 R08: 0000000000000001 R09: ffe21c006df80008
> R10: ff1100036fc0003f R11: ffe21c006df80007 R12: ff1100036fc00000
> R13: ff1100036fc000d8 R14: ff1100036fc00038 R15: ff1100036fc00000
> FS: 00007fdf1ad32740(0000) GS:ff11000e1ed00000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007fdf15f1b000 CR3: 000000011e462005 CR4: 0000000000773ee0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> PKRU: 55555554
> Call Trace:
> <TASK>
> vcpu_enter_guest+0x145d/0x24d0 [kvm]
> ? inject_pending_event+0x750/0x750 [kvm]
> ? xsaves+0x31/0x40
> ? rcu_read_lock_held_common+0x1e/0x60
> ? rcu_read_lock_sched_held+0x60/0xe0
> ? rcu_read_lock_bh_held+0xc0/0xc0
> kvm_arch_vcpu_ioctl_run+0x25d/0xcc0 [kvm]
> kvm_vcpu_ioctl+0x414/0xa30 [kvm]]
> ? kvm_clear_dirty_log_protect+0x4d0/0x4d0 [kvm]
> ? userfaultfd_unmap_prep+0x240/0x240
> ? __up_read+0x17f/0x530
> ? rwsem_wake+0x110/0x110
> ? __do_munmap+0x437/0x7c0
> ? rcu_read_lock_held_common+0x1e/0x60
> ? rcu_read_lock_sched_held+0x60/0xe0
> ? rcu_read_lock_sched_held+0x60/0xe0
> ? __kasan_check_read+0x11/0x20
> ? __fget_light+0xa9/0x100
> __x64_sys_ioctl+0xc0/0x100
> do_syscall_64+0x39/0xc0
> entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7fdf1ae493db
> Code: 0f 1e fa 48 8b 05 b5 7a 0d 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48 3d 01 f0 ff ff 73 01 c3 48 8b 0d 85 7a 0d 00 f7 d8 64 89 01 48
> RSP: 002b:00007ffcf8bdfb38 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> RAX: ffffffffffffffda RBX: 00000000006f26d0 RCX: 00007fdf1ae493db
> RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000007
> RBP: 0000000000000000 R08: 0000000000411d36 R09: 0000000000000000
> R10: fffffffffffffb69 R11: 0000000000000246 R12: 0000000000402410
> R13: 00000000006f02b0 R14: 0000000000000000 R15: 0000000000000000
> </TASK>
> irq event stamp: 0
> hardirqs last enabled at (0): [<0000000000000000>] 0x0
> hardirqs last disabled at (0): [<ffffffffb40c809a>] copy_process+0xaca/0x3270
> softirqs last enabled at (0): [<ffffffffb40c809a>] copy_process+0xaca/0x3270
> softirqs last disabled at (0): [<0000000000000000>] 0x0
> ---[ end trace 0000000000000000 ]---

The trace can be simplified to :

WARNING: arch/x86/kvm/vmx/main.c:696 vt_set_apic_access_page_addr+0x3c/0x50 [kvm_intel]
RIP: 0010:vt_set_apic_access_page_addr+0x3c/0x50 [kvm_intel]
Call Trace:
vcpu_enter_guest+0x145d/0x24d0 [kvm]
kvm_arch_vcpu_ioctl_run+0x25d/0xcc0 [kvm]
kvm_vcpu_ioctl+0x414/0xa30 [kvm]]
__x64_sys_ioctl+0xc0/0x100
do_syscall_64+0x39/0xc0
entry_SYSCALL_64_after_hwframe+0x44/0xae

Becasue here you just want to record the trace path of WARN_ON_ONCE(),
but not request some help for debugging it.

>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@xxxxxxxxx>
> ---
> arch/x86/kvm/x86.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 8f57dfb2a8c9..c90ec611de2f 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -10042,7 +10042,8 @@ void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm,
> * Update it when it becomes invalid.
> */
> apic_address = gfn_to_hva(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT);
> - if (start <= apic_address && apic_address < end)
> + if (start <= apic_address && apic_address < end &&
> + !kvm_gfn_shared_mask(kvm))

Minor: please condier to check kvm_gfn_shared_mask(kvm) before range,
means firstly check is or not, then suitable or not.

> kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD);
> }
>
> --
> 2.25.1
>