Re: [PATCH] Fix hotplug cpu on x86_64

[Brian Gerst]

Protasevich, Natalie wrote:
> > Brian Gerst wrote:
> >
> > > Brian Gerst wrote:
> > >
> > > > I've been seeing bogus values from /proc/loadavg on an x86-64 SMP 
> > > > kernel (but not UP).
> > > >
> > > > $ cat /proc/loadavg
> > > > -1012098.26 922203.26 -982431.60 1/112 2688
> > > >
> > > > This is in the current git tree.  I'm also seeing strange values in
> > > > /proc/stat:
> > > >
> > > > cpu  2489 40 920 60530 9398 171 288 1844674407350 cpu0 2509 60 940 
> > > > 60550 9418 191 308 0
> > > >
> > > > The first line is the sum of all cpus (I only have one), so it's 
> > > > picking up up bad data from the non-present cpus.  The last value, 
> > > > stolen time, is completely bogus since that value is only 
> >
> > ever used 
> >
> > > > on s390.
> > > >
> > > > It looks to me like there is some problem with how the per-cpu 
> > > > structures are being initialized, or are getting 
> >
> > corrupted.  I have 
> >
> > > > not been able to test i386 SMP yet to see if the problem is x86_64 
> > > > specific.
> > >
> > > I found the culprit: CPU hotplug.  The problem is that
> > > prefill_possible_map() is called after setup_per_cpu_areas().  This 
> > > leaves the per-cpu data sections for the future cpus uninitialized 
> > > (still pointing to the original per-cpu data, which is initmem).  
> > > Since the cpus exists in cpu_possible_map, for_each_cpu 
> >
> > will iterate 
> >
> > > over them even though the per-cpu data is invalid.
>
> I had to do the same in i386, but initially I was trying to avoid the
> whole situation - allocating per_cpu data for all possible processors.
> It seemed wasteful that on the system with NR_CPU=256 or 512 and brought
> up as 4x everything per_cpu is (pre)allocated for all, although it's
> sure convenient. I though at the time it would be great if
> alloc_percpu() mechanism was able to dynamically re-create all the
> per_cpu's for new processors, that way cpu_possible_map woun't probably
> even be needed. Or is it too much trouble for too little gain...
>
> Thanks,
> --Natalie

It certainly is possible.  In the hotplug cpu case, don't put the 
.data.percpu section in __initmem.  It will then be preserved for any 
cpus that come online after boot.

--
				Brian Gerst
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/