Re: Odd ENOMEM being returned in 3.8-rcX
From: Josh Boyer
Date: Fri Feb 08 2013 - 13:20:38 EST
On Thu, Feb 07, 2013 at 07:35:01PM -0500, Josh Boyer wrote:
> On Thu, Feb 07, 2013 at 02:15:02PM -0800, Andrew Morton wrote:
> > On Thu, 7 Feb 2013 16:57:42 -0500
> > Josh Boyer <jwboyer@xxxxxxxxxx> wrote:
> > > Hi All,
> > >
> > > We've hit a weird error in Fedora using the 3.8-rcX kernels. It seems
> > > the mock tool is getting back ENOMEM when doing very simple things that
> > > normally just work. The 3.7 kernels on the same userspace work just
> > > fine. It seems just running 'mock init -v' is enough to cause the
> > > failure.
> > I assume you're not seeing the "page allocation failure" message and
> > backtrace. This means that either
> Right. If I disable our debug options, I see no backtraces at all and
> the python app still gets ENOMEM returned. (See below for those
> > a) it's a __GFP_NOWARN callsite. This is rare. Or
> > b) it's actually a different error but someone went and overwrote a
> > callee's return value with -ENOMEM. We do this a lot and it sucks.
> We do it in copy_io :\.
> > > At first glance it seems copy_io is failing (possibly because
> > > get_task_io_context fails), and then the above fallout is printed. The
> > > warning seems fairly valid, but I don't think that is the root of the
> > > problem.
> > yes, get_task_io_context() might be the place. Tried adding a few
> > error-path printks in there to see what's happening?
> Yeah, that's my next step. I guess I know what I'll be doing tomorrow.
> > I can't see anything around there which leaves interrupts disabled
> > though. It's quite likely that there's some code with is forgetting to
> > reenable interrupts on a rarely-tested error path, and that ENOMEM is
> > tickling the bug.
> Right, agreed. As I said, I think that is mostly a secondary issue.
> Hopefully it will be easy to fix once we figure out why we're getting
> the ENOMEM error.
> Python backtrace below. Seems to be failing on forking a umount command
> after init'ing the chroot. I can put the full output somewhere if
> people are interested.
OK. I've bisected this down to:
50804fe3737ca6a5942fdc2057a18a8141d00141 is the first bad commit
Author: Eric W. Biederman <ebiederm@xxxxxxxxxxxx>
Date: Tue Mar 2 15:41:50 2010 -0800
pidns: Support unsharing the pid namespace.
I haven't really gotten much farther than that yet, but the bisect was
pretty straight forward. Eric, is there anything specific I can gather
or do to help figure out why that is causing mock to get such a weird
error? I can provide the bisect log if you'd like.
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/