Re: [PATCH 3/4] interconnect: qcom: icc-rpmh: Ensure floor BW is enforced for all nodes

[Mike Tipton]

On 7/1/2021 11:48 AM, okukatla@xxxxxxxxxxxxxx wrote:
> On 2021-06-26 02:58, Mike Tipton wrote:
> > We currently only enforce BW floors for a subset of nodes in a path.
> > All BCMs that need updating are queued in the pre_aggregate/aggregate
> > phase. The first set() commits all queued BCMs and subsequent set()
> > calls short-circuit without committing anything. Since the floor BW
> > isn't set in sum_avg/max_peak until set(), then some BCMs are committed
> > before their associated nodes reflect the floor.
> >
> > Set the floor as each node is being aggregated. This ensures that all
> > all relevant floors are set before the BCMs are committed.
> >
> > Fixes: 266cd33b5913 ("interconnect: qcom: Ensure that the floor
> > bandwidth value is enforced")
> > Signed-off-by: Mike Tipton <mdtipton@xxxxxxxxxxxxxx>
> > ---
> >  drivers/interconnect/qcom/icc-rpmh.c | 10 +++++-----
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/interconnect/qcom/icc-rpmh.c
> > b/drivers/interconnect/qcom/icc-rpmh.c
> > index bf01d09dba6c..f118f57eae37 100644
> > --- a/drivers/interconnect/qcom/icc-rpmh.c
> > +++ b/drivers/interconnect/qcom/icc-rpmh.c
> > @@ -57,6 +57,11 @@ int qcom_icc_aggregate(struct icc_node *node, u32
> > tag, u32 avg_bw,
> >              qn->sum_avg[i] += avg_bw;
> >              qn->max_peak[i] = max_t(u32, qn->max_peak[i], peak_bw);
> >          }
> > +
> > +        if (node->init_avg || node->init_peak) {
> > +            qn->sum_avg[i] = max_t(u64, qn->sum_avg[i], node->init_avg);
> > +            qn->max_peak[i] = max_t(u64, qn->max_peak[i], 
> > node->init_peak);
> > +        }
> Hi Mike,
> Original problem is BCMs not getting added to commit_list for unused 
> nodes, right? that is solved by moving *_bcm_voter_add() to 
> pre_aggregate().
> I could not get why we need to do above change, we are enforcing node 
> votes with floor votes in framework + below code snippet that you removed.
> How would adding this code in qcom_icc_aggregate() make difference? Is 
> there any other issue that i am not to able to get?

This series fixes a couple of separate issues. One is not removing the 
initial floor after sync_state for unvoted paths, which the change to 
add BCMs to the commit list in pre_aggregate addresses. The other issue 
is not properly enforcing the initial floor *before* sync_state, which 
this patch addresses.

We only commit to HW what we've aggregated in our internal, 
provider-specific buckets (AMC, WAKE, SLEEP) during 
pre_aggregate/aggregate. All BCMs in the path are added to the commit 
list during this stage. Everything in the commit list is voted on the 
*first* set() callback for the path. The commit list is empty for every 
subsequent set() callback, so nothing actually happens except in the 
first set().

The original snippet below in qcom_icc_set() partially re-aggregates AMC 
with BW values from the icc_node struct. This is to capture the floor 
enforced by the framework itself. However, this is only for a single 
node in the path, which means it's also only for a single BCM in the 
path. Since we commit all BCMs on the first set(), this means most BCMs 
in the path don't see the floor BW at the time we commit them. This can 
lead to some BCMs being under-voted, or in some cases completely off.

So, instead of aggregating the floor for each node in qcom_icc_set(), 
this patch moves it to qcom_icc_aggregate(). This means all floors are 
captured internally before committing the BCMs.

> >      }
> >
> >      *agg_avg += avg_bw;
> > @@ -90,11 +95,6 @@ int qcom_icc_set(struct icc_node *src, struct 
> > icc_node *dst)
> >      qp = to_qcom_provider(node->provider);
> >      qn = node->data;
> >
> > -    qn->sum_avg[QCOM_ICC_BUCKET_AMC] = max_t(u64,
> > qn->sum_avg[QCOM_ICC_BUCKET_AMC],
> > -                         node->avg_bw);
> > -    qn->max_peak[QCOM_ICC_BUCKET_AMC] = max_t(u64,
> > qn->max_peak[QCOM_ICC_BUCKET_AMC],
> > -                          node->peak_bw);
> > -
> >      qcom_icc_bcm_voter_commit(qp->voter);
> >
> >      return 0;