Re: [RFC PATCH 0/2] x86: Fix missing core serialization on migration

[Daniel Bristot de Oliveira]

On 11/14/2017 05:31 PM, Peter Zijlstra wrote:
> On Tue, Nov 14, 2017 at 08:16:09AM -0800, Andy Lutomirski wrote:
>> What guarantees that there's an IPI?  Do we never do a syscall, get
>> migrated during syscall processing (due to cond_resched(), for
>> example), and land on another CPU that just happened to already be
>> scheduling?
> 
> Possible, the other CPU could've pulled the task because it went idle.
> No IPIs involved in that scenario.
> 
> And if it was running a different thread of the same process prior to
> that, we'll also not do switch_mm().
> 
> So yes, it is possible to construct a migration scenario without core
> serializing instructions (of the CPUID/MOV-CR kind, not the LOCK prefix
> kind).
> 
> Note that that still requires a multi-threaded process.
> 
> There is another scenario; where the NOHZ load-balancer moves the task;
> such that the NOHZ load balancing CPU is a 3rd CPU. In that case there
> is an interrupt (to affect the load-balancing) but it will not land on
> the CPU that's going to run the task.
> 
> This could happen for a single threaded task; since I suppose the NOHZ
> idle CPU that's going to be the victim could have ran our task last and
> still lazily have the mm.
> 
> Very tricky to make work, not to mention that I suspect actually going
> idle will kill a whole bunch of state real quick.
> 

IIRC, if the dest cpu is idle and the system is with idle=poll, no IPI
is fired as well, but that is not a very common case.

-- Daniel