Re: [Fastboot] Re: kexec "problem" [and patch updates]

[Randy.Dunlap]

On Thu, 4 Mar 2004 18:33:10 +0530 Hariprasad Nellitheertha wrote:

| Hello,
| 
| I recreated this on a UNI system running an SMP kernel as well. 
| 
| The problem is because we now initialize cpu_vm_mask for init_mm with 
| CPU_MASK_ALL (from 2.6.3 onwards) which makes all bits in cpumask 1. 
| Hence BUG_ON(!cpus_equal(cpumask,tmp) fails. The change to set 
| cpu_vm_mask to CPU_MASK_ALL was done to remove tlb flush optimizations 
| for ppc64. On UNI kernels, CPU_MASK_ALL is 1 and hence the problem 
| does not occur.
| 
| I made a small patch which fixes this problem. The change is, essentially,
| to use "tmp" instead of "cpumask". This ensures that only the (other) online 
| cpus are sent the IPI. 
| 
| I have done some testing with this patch. Kexec loads fine and I haven't seen
| anything untoward. 

Yes, that does work well... Thanks for the patch.

Is this satisfactory for pushing into the mainline kernel, or
should kexec use another method to solve this problem?

--
~Randy


| Comments please.
| 
| Regards, Hari
| 
| 
| diff -Naur linux-2.6.3-before/arch/i386/kernel/smp.c linux-2.6.3/arch/i386/kernel/smp.c
| --- linux-2.6.3-before/arch/i386/kernel/smp.c	2004-02-18 09:27:15.000000000 +0530
| +++ linux-2.6.3/arch/i386/kernel/smp.c	2004-03-04 14:16:43.000000000 +0530
| @@ -356,7 +356,8 @@
|  	BUG_ON(cpus_empty(cpumask));
|  
|  	cpus_and(tmp, cpumask, cpu_online_map);
| -	BUG_ON(!cpus_equal(cpumask, tmp));
| +	if(cpus_empty(tmp))
| +		return;
|  	BUG_ON(cpu_isset(smp_processor_id(), cpumask));
|  	BUG_ON(!mm);
|  
| @@ -371,12 +372,12 @@
|  	flush_mm = mm;
|  	flush_va = va;
|  #if NR_CPUS <= BITS_PER_LONG
| -	atomic_set_mask(cpumask, &flush_cpumask);
| +	atomic_set_mask(tmp, &flush_cpumask);
|  #else
|  	{
|  		int k;
|  		unsigned long *flush_mask = (unsigned long *)&flush_cpumask;
| -		unsigned long *cpu_mask = (unsigned long *)&cpumask;
| +		unsigned long *cpu_mask = (unsigned long *)&tmp;
|  		for (k = 0; k < BITS_TO_LONGS(NR_CPUS); ++k)
|  			atomic_set_mask(cpu_mask[k], &flush_mask[k]);
|  	}
| @@ -385,7 +386,7 @@
|  	 * We have to send the IPI only to
|  	 * CPUs affected.
|  	 */
| -	send_IPI_mask(cpumask, INVALIDATE_TLB_VECTOR);
| +	send_IPI_mask(tmp, INVALIDATE_TLB_VECTOR);
|  
|  	while (!cpus_empty(flush_cpumask))
|  		/* nothing. lockup detection does not belong here */
| 
| 
| On Sat, Feb 28, 2004 at 03:41:33AM -0700, Eric W. Biederman wrote:
| > "Randy.Dunlap" <rddunlap@xxxxxxxx> writes:
| > 
| > > On 27 Feb 2004 01:00:04 -0700 Eric W. Biederman wrote:
| > > 
| > > | > It works fine on 2.6.2.  It works for me on 2.6.3 if not SMP.
| > > | > If the kernel is built for SMP, when running kexec, I get a
| > > | > BUG in arch/i386/kernel/smp.c at line 359.
| > > | > I'm testing various workarounds for that BUG now.
| > > | 
| > > | I will eyeball it...
| > > | 
| > > | Is it the kernel that is shutting down, or the kernel that is being
| > > | brought up that has problems?
| > > 
| > > the kernel that is shutting down.
| > > 
| > > | The back trace from the BUG would be interesting.
| > > 
| > > see below.  my bad.  i should have included it.
| > > 
| > > | As I see it flush_tlb_others is being called when we have shutdown
| > > | cpus and the kernel still thinks we have the mm present on foreign
| > > | cpus.
| > > 
| > > Martin Bligh thinks that there is a tlb race here.
| > > I printed the 2 cpu masks on my dual-proc macine and saw
| > > 0 in one of them and 0xc in the other one.
| > 
| > Ouch we have both cpus running when this happens, and we have not
| > started any shutdown whatsoever.  This is the bit that sets up
| > the page tables for later use...
| > 
| > I think identity_map_pages will have problems with a kernel that does
| > the 4G/4G split, and it has known issues on some other architectures,
| > because they treat init_mm specially.  So the proper solution may be
| > to simply rewrite identity_map_pages. 
| > 
| > Before we do that in the short term we need to see if
| > identity_map_pages is actually doing anything bad.  You are
| > not using the 4G/4G split so that is not the cause.  So either
| > init_mm is now special in some way, or we have hit a generic kernel
| > bug.
| > 
| > So this may indeed be a tlb race.  But it is init_mm->cpu_vm_mask and
| > cpu_online map that are different.  With the implication being
| > that init_mm->cpu_vm_mask has cpus set that are not in cpu_online_map?
| > Very weird especially on SMP.
| > 
| > Without attribution I have a hard time making sense of which cpumask
| > is which so I can't draw any conclusions.  But I find it very
| > interesting that it is bits 2 and 3 that are set.  I wonder if
| > there is any mixup between logical cpu identities and apic ids.
| > 
| > Eric
| > _______________________________________________

| 
| -- 
| Hariprasad Nellitheertha
| Linux Technology Center
| India Software Labs
| IBM India, Bangalore
| -
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/