Re: [PATCH v2 0/7] kvm: fix latent guest entry/exit bugs

From: Christian Borntraeger
Date: Wed Jan 19 2022 - 13:30:13 EST


[  457.151295] ------------[ cut here ]------------
[  457.151311] WARNING: CPU: 14 PID: 0 at kernel/rcu/tree.c:613 rcu_eqs_enter.constprop.0+0xf8/0x118
[  457.151324] Modules linked in: vhost_vsock vmw_vsock_virtio_transport_common vsock vhost vhost_iotlb xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT xt_tcpudp nft_compat nf_nat_tftp nft_objref nf_conntrack_tftp nft_counter kvm nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink sunrpc mlx5_ib ib_uverbs s390_trng ib_core genwqe_card crc_itu_t vfio_ccw mdev vfio_iommu_type1 eadm_sch vfio zcrypt_cex4 sch_fq_codel configfs ip_tables x_tables mlx5_core ghash_s390 prng aes_s390 des_s390 libdes sha3_512_s390 sha3_256_s390 sha512_s390 sha256_s390 sha1_s390 sha_common pkey zcrypt rng_core autofs4
[  457.151422] CPU: 14 PID: 0 Comm: swapper/14 Not tainted 5.16.0-00007-g89e9021389e2 #3
[  457.151428] Hardware name: IBM 3906 M04 704 (LPAR)
[  457.151432] Krnl PSW : 0404d00180000000 00000000a7c0495c (rcu_eqs_enter.constprop.0+0xfc/0x118)
[  457.151440]            R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:1 PM:0 RI:0 EA:3
[  457.151445] Krnl GPRS: ffffffffebd81d31 4000000000000000 0000000000000070 00000000a7fd7024
[  457.151450]            0000000000000000 0000000000000001 0000000000000000 0000000000000000
[  457.151454]            000000000000000e 000000000000000e 00000000a84d3a88 0000001fd8645c00
[  457.151458]            0000000000000000 0000000000000000 00000000a7c04882 0000038000653dc0
[  457.151468] Krnl Code: 00000000a7c0494c: ebaff0a00004    lmg    %r10,%r15,160(%r15)
                          00000000a7c04952: c0f4fffffef7    brcl    15,00000000a7c04740
                         #00000000a7c04958: af000000        mc    0,0
                         >00000000a7c0495c: a7f4ffa3        brc    15,00000000a7c048a2
                          00000000a7c04960: c0e500003f70    brasl    %r14,00000000a7c0c840
                          00000000a7c04966: a7f4ffcd        brc    15,00000000a7c04900
                          00000000a7c0496a: c0e500003f6b    brasl    %r14,00000000a7c0c840
                          00000000a7c04970: a7f4ffde        brc    15,00000000a7c0492c
[  457.151527] Call Trace:
[  457.151530]  [<00000000a7c0495c>] rcu_eqs_enter.constprop.0+0xfc/0x118
[  457.151536] ([<00000000a7c04882>] rcu_eqs_enter.constprop.0+0x22/0x118)
[  457.151540]  [<00000000a7c14cd2>] default_idle_call+0x62/0xd8
[  457.151545]  [<00000000a6f816c6>] do_idle+0xf6/0x1b0
[  457.151553]  [<00000000a6f81a06>] cpu_startup_entry+0x36/0x40
[  457.151558]  [<00000000a7c16abe>] restart_int_handler+0x6e/0x90
[  457.151563] no locks held by swapper/14/0.
[  457.151567] Last Breaking-Event-Address:
[  457.151570]  [<00000000a7c0489e>] rcu_eqs_enter.constprop.0+0x3e/0x118
[  457.151574] irq event stamp: 608654
[  457.151578] hardirqs last  enabled at (608653): [<00000000a70190d8>] tick_nohz_idle_enter+0xb0/0x130
[  457.151584] hardirqs last disabled at (608654): [<00000000a6f8173e>] do_idle+0x16e/0x1b0
[  457.151589] softirqs last  enabled at (608586): [<00000000a7c1861a>] __do_softirq+0x4ba/0x668
[  457.151594] softirqs last disabled at (608581): [<00000000a6f367c6>] __irq_exit_rcu+0x13e/0x170
[  457.151600] ---[ end trace 2ae2154f9724de86 ]---

I can not see right now whats wrong, your patches look sane.


Now followed by (might be a followup to the error above).

[ 574.301736] =============================
[ 574.301741] WARNING: suspicious RCU usage
[ 574.301746] 5.16.0-00007-g89e9021389e2 #3 Tainted: G W
[ 574.301751] -----------------------------
[ 574.301755] kernel/rcu/tree.c:821 Bad RCU dynticks_nmi_nesting counter
!
[ 574.301760]
other info that might help us debug this:

[ 574.301764]
rcu_scheduler_active = 2, debug_locks = 1
[ 574.301769] 2 locks held by CPU 0/KVM/8327:
[ 574.301773] #0: 00000001521956b8 (&vcpu->mutex){+.+.}-{3:3}, at: kvm_vcpu_ioctl+0x9a/0xa40 [kvm]
[ 574.301843] #1: 000000016e92c6a8 (&kvm->srcu){....}-{0:0}, at: __vcpu_run+0x332/0x5b8 [kvm]
[ 574.301873]
stack backtrace:
[ 574.301878] CPU: 46 PID: 8327 Comm: CPU 0/KVM Tainted: G W 5.16.0-00007-g89e9021389e2 #3
[ 574.301884] Hardware name: IBM 3906 M04 704 (LPAR)
[ 574.301888] Call Trace:
[ 574.301892] [<00000000a7c001c6>] dump_stack_lvl+0x8e/0xc8
[ 574.301903] [<00000000a6fee70e>] rcu_irq_exit_check_preempt+0x136/0x1c8
[ 574.301913] [<00000000a6ff9d1a>] irqentry_exit_cond_resched+0x32/0x80
[ 574.301921] [<00000000a7c0521c>] irqentry_exit+0xac/0x130
[ 574.301927] [<00000000a7c16626>] ext_int_handler+0xe6/0x120
[ 574.301933] [<00000000a6fc2482>] lock_acquire.part.0+0x12a/0x258
[ 574.301939] ([<00000000a6fc2450>] lock_acquire.part.0+0xf8/0x258)
[ 574.301944] [<00000000a6fc2660>] lock_acquire+0xb0/0x200
[ 574.302053] [<000003ff807092ce>] __vcpu_run+0x376/0x5b8 [kvm]
[ 574.302076] [<000003ff807099ba>] kvm_arch_vcpu_ioctl_run+0x10a/0x270 [kvm]
[ 574.302098] [<000003ff806f002c>] kvm_vcpu_ioctl+0x27c/0xa40 [kvm]
[ 574.302120] [<00000000a728b5b6>] __s390x_sys_ioctl+0xbe/0x100
[ 574.302129] [<00000000a7c038ea>] __do_syscall+0x1da/0x208
[ 574.302133] [<00000000a7c16352>] system_call+0x82/0xb0
[ 574.302138] 2 locks held by CPU 0/KVM/8327:
[ 574.302143] #0: 00000001521956b8 (&vcpu->mutex){+.+.}-{3:3}, at: kvm_vcpu_ioctl+0x9a/0xa40 [kvm]
[ 574.302172] #1: 000000016e92c6a8 (&kvm->srcu){....}-{0:0}, at: __vcpu_run+0x332/0x5b8 [kvm]