Re: [PATCH 2/2] iommu/amd: Remove performance counter pre-initialization test

From: David Coe
Date: Wed Apr 14 2021 - 11:33:45 EST

Next message: David Laight: "RE: [PATCH 1/2] audit: Add syscall return code handling for compat task"
Previous message: Marc Kleine-Budde: "Re: [PATCH v2 3/6] dt-bindings: phy: Add binding for TI TCAN104x CAN transceivers"
In reply to: Suthikulpanit, Suravee: "Re: [PATCH 2/2] iommu/amd: Remove performance counter pre-initialization test"
Next in thread: Suthikulpanit, Suravee: "Re: [PATCH 2/2] iommu/amd: Remove performance counter pre-initialization test"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi Suravee!

I've re-run your revert+update patch on Ubuntu's latest kernel 5.11.0-14 partly to check my mailer's 'mangling' hadn't also reached the code!

There are 3 sets of results in the attachment, all for the Ryzen 2400G. The as-distributed kernel already incorporates your IOMMU RFCv3 patch.

A. As-distributed kernel (cold boot)
>5 retries, so no IOMMU read/write capability, no amd_iommu events.

B. As-distributed kernel (warm boot)
<5 retries, amd_iommu running stats show large numbers as before.

C. Revert+Update kernel
amd_iommu events listed and also show large hit/miss numbers.

In due course, I'll load the new (revert+update) kernel on the 4700G but won't overload your mail-box unless something unusual turns up.

Best regards,

--
David
A. As Supplied - Cold Boot
**************************

$ sudo dmesg | grep IOMMU
[ 0.710610] pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter.
[ 0.714365] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.984616] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@xxxxxxx>

$ sudo perf list | grep iommu
intel_iommu:bounce_map_sg [Tracepoint event]
intel_iommu:bounce_map_single [Tracepoint event]
intel_iommu:bounce_unmap_single [Tracepoint event]
intel_iommu:map_sg [Tracepoint event]
intel_iommu:map_single [Tracepoint event]
intel_iommu:unmap_sg [Tracepoint event]
intel_iommu:unmap_single [Tracepoint event]
iommu:add_device_to_group [Tracepoint event]
iommu:attach_device_to_domain [Tracepoint event]
iommu:detach_device_from_domain [Tracepoint event]
iommu:io_page_fault [Tracepoint event]
iommu:map [Tracepoint event]
iommu:remove_device_from_group [Tracepoint event]
iommu:unmap [Tracepoint event]

No amd_iommu events listed.

B. As Supplied - Warm Boot
**************************

$ sudo dmesg | grep IOMMU
[ 0.515523] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.519236] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.520549] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 0.795781] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@xxxxxxx>

$ sudo perf stat -e 'amd_iommu_0/cmd_processed/, amd_iommu_0/cmd_processed_inv/, amd_iommu_0/ign_rd_wr_mmio_1ff8h/, amd_iommu_0/int_dte_hit/, amd_iommu_0/int_dte_mis/, amd_iommu_0/mem_dte_hit/, amd_iommu_0/mem_dte_mis/, amd_iommu_0/mem_iommu_tlb_pde_hit/, amd_iommu_0/mem_iommu_tlb_pde_mis/, amd_iommu_0/mem_iommu_tlb_pte_hit/, amd_iommu_0/mem_iommu_tlb_pte_mis/, amd_iommu_0/mem_pass_excl/, amd_iommu_0/mem_pass_pretrans/, amd_iommu_0/mem_pass_untrans/, amd_iommu_0/mem_target_abort/, amd_iommu_0/mem_trans_total/, amd_iommu_0/page_tbl_read_gst/, amd_iommu_0/page_tbl_read_nst/, amd_iommu_0/page_tbl_read_tot/, amd_iommu_0/smi_blk/, amd_iommu_0/smi_recv/, amd_iommu_0/tlb_inv/, amd_iommu_0/vapic_int_guest/, amd_iommu_0/vapic_int_non_guest/' sleep 10

Performance counter stats for 'system wide':

0 amd_iommu_0/cmd_processed/ (33.39%)
0 amd_iommu_0/cmd_processed_inv/ (33.06%)
0 amd_iommu_0/ign_rd_wr_mmio_1ff8h/ (33.58%)
842,245,888,947,849 amd_iommu_0/int_dte_hit/ (33.42%)
848,982,159,636,118 amd_iommu_0/int_dte_mis/ (33.15%)
835,698,854,752,581 amd_iommu_0/mem_dte_hit/ (33.68%)
839,060,819,932,270 amd_iommu_0/mem_dte_mis/ (33.55%)
0 amd_iommu_0/mem_iommu_tlb_pde_hit/ (33.44%)
837,231,240,047,576 amd_iommu_0/mem_iommu_tlb_pde_mis/ (33.62%)
842,688,371,629,123 amd_iommu_0/mem_iommu_tlb_pte_hit/ (33.40%)
851,647,568,857,291 amd_iommu_0/mem_iommu_tlb_pte_mis/ (33.05%)
0 amd_iommu_0/mem_pass_excl/ (33.30%)
0 amd_iommu_0/mem_pass_pretrans/ (33.36%)
852,801,037,224,491 amd_iommu_0/mem_pass_untrans/ (33.01%)
0 amd_iommu_0/mem_target_abort/ (33.50%)
46,371 amd_iommu_0/mem_trans_total/ (33.28%)
0 amd_iommu_0/page_tbl_read_gst/ (33.00%)
1,663 amd_iommu_0/page_tbl_read_nst/ (33.55%)
17 amd_iommu_0/page_tbl_read_tot/ (33.38%)
0 amd_iommu_0/smi_blk/ (33.28%)
0 amd_iommu_0/smi_recv/ (33.49%)
0 amd_iommu_0/tlb_inv/ (33.32%)
0 amd_iommu_0/vapic_int_guest/ (32.96%)
318 amd_iommu_0/vapic_int_non_guest/ (33.28%)

10.001298695 seconds time elapsed

C. Revert + Update
******************

$ sudo dmesg | grep IOMMU
[ 0.557917] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.561705] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.563004] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 0.905418] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@xxxxxxx>

$ sudo perf list | grep iommu
amd_iommu_0/cmd_processed/ [Kernel PMU event]
amd_iommu_0/cmd_processed_inv/ [Kernel PMU event]
amd_iommu_0/ign_rd_wr_mmio_1ff8h/ [Kernel PMU event]
amd_iommu_0/int_dte_hit/ [Kernel PMU event]
amd_iommu_0/int_dte_mis/ [Kernel PMU event]
amd_iommu_0/mem_dte_hit/ [Kernel PMU event]
amd_iommu_0/mem_dte_mis/ [Kernel PMU event]
amd_iommu_0/mem_iommu_tlb_pde_hit/ [Kernel PMU event]
amd_iommu_0/mem_iommu_tlb_pde_mis/ [Kernel PMU event]
amd_iommu_0/mem_iommu_tlb_pte_hit/ [Kernel PMU event]
amd_iommu_0/mem_iommu_tlb_pte_mis/ [Kernel PMU event]
amd_iommu_0/mem_pass_excl/ [Kernel PMU event]
amd_iommu_0/mem_pass_pretrans/ [Kernel PMU event]
amd_iommu_0/mem_pass_untrans/ [Kernel PMU event]
amd_iommu_0/mem_target_abort/ [Kernel PMU event]
amd_iommu_0/mem_trans_total/ [Kernel PMU event]
amd_iommu_0/page_tbl_read_gst/ [Kernel PMU event]
amd_iommu_0/page_tbl_read_nst/ [Kernel PMU event]
amd_iommu_0/page_tbl_read_tot/ [Kernel PMU event]
amd_iommu_0/smi_blk/ [Kernel PMU event]
amd_iommu_0/smi_recv/ [Kernel PMU event]
amd_iommu_0/tlb_inv/ [Kernel PMU event]
amd_iommu_0/vapic_int_guest/ [Kernel PMU event]
amd_iommu_0/vapic_int_non_guest/ [Kernel PMU event]
intel_iommu:bounce_map_sg [Tracepoint event]
intel_iommu:bounce_map_single [Tracepoint event]
intel_iommu:bounce_unmap_single [Tracepoint event]
intel_iommu:map_sg [Tracepoint event]
intel_iommu:map_single [Tracepoint event]
intel_iommu:unmap_sg [Tracepoint event]
intel_iommu:unmap_single [Tracepoint event]
iommu:add_device_to_group [Tracepoint event]
iommu:attach_device_to_domain [Tracepoint event]
iommu:detach_device_from_domain [Tracepoint event]
iommu:io_page_fault [Tracepoint event]
iommu:map [Tracepoint event]
iommu:remove_device_from_group [Tracepoint event]
iommu:unmap [Tracepoint event]

$ sudo perf stat -e 'amd_iommu_0/cmd_processed/, amd_iommu_0/cmd_processed_inv/, amd_iommu_0/ign_rd_wr_mmio_1ff8h/, amd_iommu_0/int_dte_hit/, amd_iommu_0/int_dte_mis/, amd_iommu_0/mem_dte_hit/, amd_iommu_0/mem_dte_mis/, amd_iommu_0/mem_iommu_tlb_pde_hit/, amd_iommu_0/mem_iommu_tlb_pde_mis/, amd_iommu_0/mem_iommu_tlb_pte_hit/, amd_iommu_0/mem_iommu_tlb_pte_mis/, amd_iommu_0/mem_pass_excl/, amd_iommu_0/mem_pass_pretrans/, amd_iommu_0/mem_pass_untrans/, amd_iommu_0/mem_target_abort/, amd_iommu_0/mem_trans_total/, amd_iommu_0/page_tbl_read_gst/, amd_iommu_0/page_tbl_read_nst/, amd_iommu_0/page_tbl_read_tot/, amd_iommu_0/smi_blk/, amd_iommu_0/smi_recv/, amd_iommu_0/tlb_inv/, amd_iommu_0/vapic_int_guest/, amd_iommu_0/vapic_int_non_guest/' sleep 10

Performance counter stats for 'system wide':

36 amd_iommu_0/cmd_processed/ (33.30%)
20 amd_iommu_0/cmd_processed_inv/ (33.34%)
0 amd_iommu_0/ign_rd_wr_mmio_1ff8h/ (33.38%)
842,158,150,822,877 amd_iommu_0/int_dte_hit/ (33.42%)
0 amd_iommu_0/int_dte_mis/ (33.44%)
310 amd_iommu_0/mem_dte_hit/ (33.44%)
841,709,432,950,259 amd_iommu_0/mem_dte_mis/ (33.44%)
0 amd_iommu_0/mem_iommu_tlb_pde_hit/ (33.44%)
320 amd_iommu_0/mem_iommu_tlb_pde_mis/ (33.42%)
182 amd_iommu_0/mem_iommu_tlb_pte_hit/ (33.38%)
10,863 amd_iommu_0/mem_iommu_tlb_pte_mis/ (33.34%)
0 amd_iommu_0/mem_pass_excl/ (33.30%)
0 amd_iommu_0/mem_pass_pretrans/ (33.28%)
10,834 amd_iommu_0/mem_pass_untrans/ (33.28%)
0 amd_iommu_0/mem_target_abort/ (33.28%)
1,430 amd_iommu_0/mem_trans_total/ (33.28%)
0 amd_iommu_0/page_tbl_read_gst/ (33.28%)
201 amd_iommu_0/page_tbl_read_nst/ (33.28%)
192 amd_iommu_0/page_tbl_read_tot/ (33.28%)
0 amd_iommu_0/smi_blk/ (33.28%)
0 amd_iommu_0/smi_recv/ (33.28%)
0 amd_iommu_0/tlb_inv/ (33.28%)
0 amd_iommu_0/vapic_int_guest/ (33.28%)
336 amd_iommu_0/vapic_int_non_guest/ (33.28%)

10.001183400 seconds time elapsed

Next message: David Laight: "RE: [PATCH 1/2] audit: Add syscall return code handling for compat task"
Previous message: Marc Kleine-Budde: "Re: [PATCH v2 3/6] dt-bindings: phy: Add binding for TI TCAN104x CAN transceivers"
In reply to: Suthikulpanit, Suravee: "Re: [PATCH 2/2] iommu/amd: Remove performance counter pre-initialization test"
Next in thread: Suthikulpanit, Suravee: "Re: [PATCH 2/2] iommu/amd: Remove performance counter pre-initialization test"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]