Re: [RESEND PATCH v17 2/5] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device

[Vivek Gautam]

On 11/26/2018 11:33 AM, Vivek Gautam wrote:
> On 11/24/2018 12:06 AM, Will Deacon wrote:
> > On Thu, Nov 22, 2018 at 05:32:24PM +0530, Vivek Gautam wrote:
> > > Hi Will,
> > >
> > > On Wed, Nov 21, 2018 at 11:09 PM Will Deacon <will.deacon@xxxxxxx> 
> > > wrote:
> > > > On Fri, Nov 16, 2018 at 04:54:27PM +0530, Vivek Gautam wrote:
> > > > > From: Sricharan R <sricharan@xxxxxxxxxxxxxx>
> > > > >
> > > > > The smmu device probe/remove and add/remove master device callbacks
> > > > > gets called when the smmu is not linked to its master, that is 
> > > > > without
> > > > > the context of the master device. So calling runtime apis in those 
> > > > > places
> > > > > separately.
> > > > > Global locks are also initialized before enabling runtime pm as the
> > > > > runtime_resume() calls device_reset() which does tlb_sync_global()
> > > > > that ultimately requires locks to be initialized.
> > > > >
> > > > > Signed-off-by: Sricharan R <sricharan@xxxxxxxxxxxxxx>
> > > > > [vivek: Cleanup pm runtime calls]
> > > > > Signed-off-by: Vivek Gautam <vivek.gautam@xxxxxxxxxxxxxx>
> > > > > Reviewed-by: Tomasz Figa <tfiga@xxxxxxxxxxxx>
> > > > > Tested-by: Srinivas Kandagatla <srinivas.kandagatla@xxxxxxxxxx>
> > > > > Reviewed-by: Robin Murphy <robin.murphy@xxxxxxx>
> > > > > ---
> > > > > Â drivers/iommu/arm-smmu.c | 101 
> > > > > ++++++++++++++++++++++++++++++++++++++++++-----
> > > > > Â 1 file changed, 91 insertions(+), 10 deletions(-)
> > > > Given that you're doing the get/put in the TLBI ops unconditionally:
> > > >
> > > > > Â static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain)
> > > > > Â {
> > > > > ÂÂÂÂÂÂ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
> > > > > +ÂÂÂÂ struct arm_smmu_device *smmu = smmu_domain->smmu;
> > > > >
> > > > > -ÂÂÂÂ if (smmu_domain->tlb_ops)
> > > > > +ÂÂÂÂ if (smmu_domain->tlb_ops) {
> > > > > +ÂÂÂÂÂÂÂÂÂÂÂÂ arm_smmu_rpm_get(smmu);
> > > > > smmu_domain->tlb_ops->tlb_flush_all(smmu_domain);
> > > > > +ÂÂÂÂÂÂÂÂÂÂÂÂ arm_smmu_rpm_put(smmu);
> > > > > +ÂÂÂÂ }
> > > > > Â }
> > > > >
> > > > > Â static void arm_smmu_iotlb_sync(struct iommu_domain *domain)
> > > > > Â {
> > > > > ÂÂÂÂÂÂ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
> > > > > +ÂÂÂÂ struct arm_smmu_device *smmu = smmu_domain->smmu;
> > > > >
> > > > > -ÂÂÂÂ if (smmu_domain->tlb_ops)
> > > > > +ÂÂÂÂ if (smmu_domain->tlb_ops) {
> > > > > +ÂÂÂÂÂÂÂÂÂÂÂÂ arm_smmu_rpm_get(smmu);
> > > > > smmu_domain->tlb_ops->tlb_sync(smmu_domain);
> > > > > +ÂÂÂÂÂÂÂÂÂÂÂÂ arm_smmu_rpm_put(smmu);
> > > > > +ÂÂÂÂ }
> > > > Why do you need them around the map/unmap calls as well?
> > > We still have .tlb_add_flush path?
> > Ok, so we could add the ops around that as well. Right now, we've got
> > the runtime pm hooks crossing two parts of the API.
>
> Sure, will do that then, and remove the runtime pm hooks from map/unmap.

I missed this earlier -
We are adding runtime pm hooks in the 'iommu_ops' callbacks and not 
really to
'tlb_ops'. So how the runtime pm hooks crossing the paths?
'.map/.unmap' iommu_ops don't call '.flush_iotlb_all' or '.iotlb_sync' 
iommu_ops
anywhere.

E.g., only callers to domain->ops->flush_iotlb_all() are:
iommu_dma_flush_iotlb_all(), or iommu_flush_tlb_all() which are not in 
map/unmap paths.

Regards
Vivek

> Thanks
> Vivek
> > Will