Re: [PATCH v2] [RFC PATCH] PCI: Update LTR threshold based on LTRME bit

From: Bjorn Helgaas
Date: Tue Apr 12 2022 - 19:22:21 EST


[+cc Vidya, Kenny]

On Tue, Mar 08, 2022 at 12:29:09AM +0530, Prasad Malisetty wrote:
> Update LTR threshold scale and value based on LTRME (Latency
> Tolenrance Reporting Mechanism) from device capabilities.
>
> In ASPM driver, LTR threshold scale and value is updating
> based on tcommon_mode and t_poweron values. In kioxia NVMe,
> L1.2 is failing due to LTR threshold scale and value is
> greater values than max snoop/non snoop value.
>
> In general, updated LTR threshold scale and value should be
> less than max snoop/non snoop value to enter the device
> into L1.2 state.

Interesting that you mention an L1.2 issue on a KIOXIA NVMe device.

Kenny also reported an L1 Substates issue related to a KIOXIA NVMe
device at [1]. That issue happened when saving/restoring the L1 SS
state for suspend/resume.

We ended up reverting 4257f7e008ea to avoid the problem, but when he
tested that commit later, the issue did not occur [2].

I don't know if there's a connection here, so this is just a heads-up
in case there is.

[1] https://lore.kernel.org/r/20201228040513.GA611645@bjorn-Precision-5520
[2] https://lore.kernel.org/r/3ca14a7-b726-8430-fe61-a3ac183a1088@xxxxxxxxx

> Signed-off-by: Prasad Malisetty <quic_pmaliset@xxxxxxxxxxx>
>
> ---
> Changes since v1:
> - Added missing variable declaration in v1 patch.
> ---
> drivers/pci/pcie/aspm.c | 12 +++++++++---
> 1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a96b742..a67746c 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -463,6 +463,7 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
> u32 val1, val2, scale1, scale2;
> u32 t_common_mode, t_power_on, l1_2_threshold, scale, value;
> u32 ctl1 = 0, ctl2 = 0;
> + u32 cap;
> u32 pctl1, pctl2, cctl1, cctl2;
> u32 pl1_2_enables, cl1_2_enables;
>
> @@ -499,9 +500,14 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
> * Table 5-11. T(POWER_OFF) is at most 2us and T(L1.2) is at
> * least 4us.
> */
> - l1_2_threshold = 2 + 4 + t_common_mode + t_power_on;
> - encode_l12_threshold(l1_2_threshold, &scale, &value);
> - ctl1 |= t_common_mode << 8 | scale << 29 | value << 16;
> + pcie_capability_read_dword(child, PCI_EXP_DEVCAP2, &cap);
> + if (!(cap & PCI_EXP_DEVCAP2_LTR)) {
> + l1_2_threshold = 2 + 4 + t_common_mode + t_power_on;
> + encode_l12_threshold(l1_2_threshold, &scale, &value);
> + ctl1 |= scale << 29 | value << 16;
> + }
> +
> + ctl1 |= t_common_mode;
>
> /* Some broken devices only support dword access to L1 SS */
> pci_read_config_dword(parent, parent->l1ss + PCI_L1SS_CTL1, &pctl1);
> --
> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
> of Code Aurora Forum, hosted by The Linux Foundation
>