Re: [HMM v13 09/18] mm/hmm/mirror: mirror process address space on device with HMM helpers

[Jerome Glisse]

On Mon, Nov 21, 2016 at 01:42:43PM +1100, Balbir Singh wrote:
> On 19/11/16 05:18, Jérôme Glisse wrote:

[...]

> > +/*
> > + * hmm_mirror_register() - register a mirror against an mm
> > + *
> > + * @mirror: new mirror struct to register
> > + * @mm: mm to register against
> > + *
> > + * To start mirroring a process address space device driver must register an
> > + * HMM mirror struct.
> > + */
> > +int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm)
> > +{
> > +	/* Sanity check */
> > +	if (!mm || !mirror || !mirror->ops)
> > +		return -EINVAL;
> > +
> > +	mirror->hmm = hmm_register(mm);
> > +	if (!mirror->hmm)
> > +		return -ENOMEM;
> > +
> > +	/* Register mmu_notifier if not already, use mmap_sem for locking */
> > +	if (!mirror->hmm->mmu_notifier.ops) {
> > +		struct hmm *hmm = mirror->hmm;
> > +		down_write(&mm->mmap_sem);
> > +		if (!hmm->mmu_notifier.ops) {
> > +			hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops;
> > +			if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) {
> > +				hmm->mmu_notifier.ops = NULL;
> > +				up_write(&mm->mmap_sem);
> > +				return -ENOMEM;
> > +			}
> > +		}
> > +		up_write(&mm->mmap_sem);
> > +	}
> 
> Does everything get mirrored, every update to the PTE (clear dirty, clear
> accessed bit, etc) or does the driver decide?

Driver decide but only read/write/valid matter for device. Device driver must
report dirtyness on invalidation. Some device do not have access bit and thus
can't provide that information.

The idea here is really to snapshot the CPU page table and duplicate it as
a GPU page table. The only synchronization HMM provide is that each virtual
address point to same memory at that at no point in time the same virtual
address can point to different physical memory on the device and on the CPU.

Cheers,
Jérôme