[next-20220225][Oops][ppc] lvm snapshot merge results kernel panics (throtl_pending_timer_fn)
From: Abdul Haleem
Date: Wed Mar 02 2022 - 03:05:07 EST
Greeting's
Linux next kernel 5.17.0-rc5-next-20220225 crashed on my power 10 LPAR
when merge lvm snapshot on nvme disk
console logs
------------
fdisk -l /dev/nvme1n1
Disk /dev/nvme1n1: 372.6 GiB, 400088457216 bytes, 97677846 sectors
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
vgcreate avocado_vg /dev/nvme1n1
Volume group "avocado_vg" successfully created
lvcreate --name avocado_lv --size 18432.0 avocado_vg -y
Wiping ext2 signature on /dev/avocado_vg/avocado_lv.
Logical volume "avocado_lv" created.
lvdisplay avocado_vg
--- Logical volume ---
LV Path /dev/avocado_vg/avocado_lv
LV Name avocado_lv
VG Name avocado_vg
LV UUID nkhkFh-Oofl-GKH1-1055-3B47-0gup-yQtI1s
LV Write Access read/write
LV Creation host, time ltc-zz1b-lp4.aus.stglabs.ibm.com, 2022-03-02
01:32:19 -0600
LV Status available
# open 0
LV Size 18.00 GiB
Current LE 4608
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:0
mkfs.ext2 /dev/avocado_vg/avocado_lv
mke2fs 1.44.6 (5-Mar-2019)
Creating filesystem with 4718592 4k blocks and 1179648 inodes
Filesystem UUID: 5ed3c335-bac2-4b64-a827-222d287af0b2
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632,
2654208,
4096000
Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
mount /dev/avocado_vg/avocado_lv /mnt
[ 1053.517019] EXT4-fs (dm-0): mounting ext2 file system using the ext4
subsystem
[ 1053.518270] EXT4-fs (dm-0): mounted filesystem without journal. Quota
mode: none.
umount /dev/avocado_vg/avocado_lv
lvdisplay avocado_vg
--- Logical volume ---
LV Path /dev/avocado_vg/avocado_lv
LV Name avocado_lv
VG Name avocado_vg
LV UUID nkhkFh-Oofl-GKH1-1055-3B47-0gup-yQtI1s
LV Write Access read/write
LV Creation host, time ltc-zz1b-lp4.aus.stglabs.ibm.com, 2022-03-02
01:32:19 -0600
LV Status available
# open 0
LV Size 18.00 GiB
Current LE 4608
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:0
lvcreate --snapshot --name avocado_sn /dev/avocado_vg/avocado_lv
--ignoreactivationskip --size 18432.0
Logical volume "avocado_sn" created.
vgdisplay avocado_vg
--- Volume group ---
VG Name avocado_vg
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size <372.61 GiB
PE Size 4.00 MiB
Total PE 95388
Alloc PE / Size 9216 / 36.00 GiB
Free PE / Size 86172 / <336.61 GiB
VG UUID jobi1f-kHw4-2ovw-nR3E-Eml5-tFYR-mJc3WT
lvdisplay avocado_vg
--- Logical volume ---
LV Path /dev/avocado_vg/avocado_lv
LV Name avocado_lv
VG Name avocado_vg
LV UUID nkhkFh-Oofl-GKH1-1055-3B47-0gup-yQtI1s
LV Write Access read/write
LV Creation host, time ltc-zz1b-lp4.aus.stglabs.ibm.com, 2022-03-02
01:32:19 -0600
LV snapshot status source of
avocado_sn [active]
LV Status available
# open 0
LV Size 18.00 GiB
Current LE 4608
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--- Logical volume ---
LV Path /dev/avocado_vg/avocado_sn
LV Name avocado_sn
VG Name avocado_vg
LV UUID QL14R9-aVa9-nD8z-DTyg-87qL-b2yY-O30G5P
LV Write Access read/write
LV Creation host, time ltc-zz1b-lp4.aus.stglabs.ibm.com, 2022-03-02
01:32:20 -0600
LV snapshot status active destination for avocado_lv
LV Status available
# open 0
LV Size 18.00 GiB
Current LE 4608
COW-table size 18.00 GiB
COW-table LE 4608
Allocated to snapshot 0.00%
Snapshot chunk size 4.00 KiB
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:3
lvconvert --merge --interval 1 /dev/avocado_vg/avocado_sn
Merging of volume avocado_vg/avocado_sn started.
avocado_vg/avocado_lv: Merged: 100.00%
BUG: Unable to handle kernel data access on write at 0x3335323d524f4ab1
Faulting instruction address: 0xc000000000cb3a2c
Oops: Kernel access of bad area, sig: 11 [#1]
LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries
Modules linked in: dm_snapshot dm_bufio rpadlpar_io rpaphp kvm_pr kvm
nf_tables libcrc32c
nfnetlink tcp_diag udp_diag inet_diag unix_diag af_packet_diag
netlink_diag rfkill dm_mod sg pseries_rn
g xts uio_pdrv_genirq vmx_crypto gf128mul uio nfsd auth_rpcgss nfs_acl
lockd grace sunrpc binfmt_misc sc
h_fq_codel ip_tables ext4 mbcache jbd2 sd_mod ibmvscsi ibmvnic
scsi_transport_srp ibmveth nvme nvme_core
t10_pi
CPU: 17 PID: 0 Comm: swapper/17 Kdump: loaded Not tainted
5.17.0-rc5-next-20220225-autote
NIP: c000000000cb3a2c LR: c0000000006740b4 CTR: c000000000674030
REGS: c0000000034ff390 TRAP: 0380 Not tainted
(5.17.0-rc5-next-20220225-autotest)
MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 82008228 XER: 20040000
CFAR: c0000000006740b0 IRQMASK: 1
GPR00: c000000000220290 c0000000034ff630 c0000000028d3200
3335323d524f4ab1
GPR04: c000000000674030 00000001000126d5 0000000000000200
0000000000000000
GPR08: 0000000000000000 0000000000000000 0000000000000001
0000000000000001
GPR12: 0000000022000222 c00000000ffa6680 0000000000000100
0000000000000010
GPR16: 0000000000000100 0000000000000002 c000000002902108
c0000000020e7880
GPR20: c000000000674030 5deadbeef0000122 c00000077c908668
c0000000b4dce000
GPR24: c0000000acb6a800 c0000000034ff7c8 3335323d524f4a41
0000000000000100
GPR28: c000000000674030 c0000000acb6a880 3335323d524f4ab1
c0000000acb6a830
NIP [c000000000cb3a2c] _raw_spin_lock_irq+0x1c/0xa0
LR [c0000000006740b4] throtl_pending_timer_fn+0x84/0x450
Call Trace:
[c0000000034ff630] [c0000000034ff670] 0xc0000000034ff670 (unreliable)
[c0000000034ff650] [c000000002903a00] jiffies+0x0/0x80
[c0000000034ff710] [c000000000220290] call_timer_fn+0x50/0x200
[c0000000034ff7a0] [c000000000221c30] run_timer_softirq+0x340/0x7c0
[c0000000034ff870] [c000000000cb3cb8] __do_softirq+0x158/0x3e0
[c0000000034ff970] [c00000000014ef78] irq_exit+0x168/0x1b0
[c0000000034ff9a0] [c0000000000271c4] timer_interrupt+0x1a4/0x3e0
[c0000000034ffa00] [c000000000009a08] decrementer_common_virt+0x208/0x210
--- interrupt: 900 at plpar_hcall_norets_notrace+0x18/0x2c
NIP: c0000000000ece58 LR: c0000000009d58c8 CTR: 0000000000000000
REGS: c0000000034ffa70 TRAP: 0900 Not tainted
(5.17.0-rc5-next-20220225-autotest)
MSR: 800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR:
22000224 XER: 20040000
CFAR: 0000000000000000 IRQMASK: 0
GPR00: 0000000000000000 c0000000034ffd10 c0000000028d3200
0000000000000000
GPR04: 000000000000ffff 0000000000000000 0000000000000000
0000000000000000
GPR08: 0000000000000000 0000000000000000 800000140fc3fc00
ffffffffffffffff
GPR12: 0000000000000000 c00000000ffa6680 0000000000010000
0000000000000800
GPR16: c0000000016a20e8 c00000077fff1678 ffffffffffffffff
0000000000000001
GPR20: 0000000000000001 0000000000000002 0000000000000010
c00000000282ff88
GPR24: 0000000000000001 000000f59a421dce 0000000000000000
0000000000000000
GPR28: 0000000000000001 0000000000000000 c0000000020e2098
c0000000020e20a0
NIP [c0000000000ece58] plpar_hcall_norets_notrace+0x18/0x2c
LR [c0000000009d58c8] check_and_cede_processor+0x48/0x60
--- interrupt: 900
[c0000000034ffd10] [c0000000034ffd40] 0xc0000000034ffd40 (unreliable)
[c0000000034ffd70] [c0000000009d5d04] dedicated_cede_loop+0x94/0x1a0
[c0000000034ffdc0] [c0000000009d2634] cpuidle_enter_state+0x2d4/0x4e0
[c0000000034ffe20] [c0000000009d28d8] cpuidle_enter+0x48/0x70
[c0000000034ffe60] [c00000000019db74] call_cpuidle+0x44/0x80
[c0000000034ffe80] [c00000000019e150] do_idle+0x340/0x390
[c0000000034fff00] [c00000000019e3d4] cpu_startup_entry+0x34/0x40
[c0000000034fff30] [c0000000000607e0] start_secondary+0x290/0x2b0
[c0000000034fff90] [c00000000000d154] start_secondary_prolog+0x10/0x14
Instruction dump:
7c0803a6 4bffffa4 60000000 60000000 60000000 3c4c01c2 3842f7f0 f821ffe1
39200001 992d0932 39200000 39400001 <7c801829> 7c044800 40c20010 7d40192d
---[ end trace 0000000000000000 ]---
--
Regard's
Abdul Haleem
IBM Linux Technology Center