Re: Avoiding *mandatory* overcommit...

From: Jesse Pollard (jesse@cats-chateau.net)
Date: Tue Apr 04 2000 - 05:43:53 EST


Marco Colombo wrote:
>
> On Mon, 3 Apr 2000, Linda Walsh wrote:
>
> > Marco Colombo wrote:
> > > It's not a matter of how I call it. The kernel gives you addresses, which
> > > you are able to use. You read at a given address, you get data. You
> > > write to an address, and you ask the kernel to allocate memory. That
> > > request may fail.
> > ---
> > This would be fine for an "address manager", but the Linux kernel has
> > something called a "memory manager". What kernel are you talking about?
>
> Nice game. Once more "names" are different from "objects".
> Whatever you name it brk() just modifies the address space. No memory
> management is performed. So it's not part of Linux mm. How it's
> implemented in other ancient OSes, I don't care...
> What kernel are *you* talking about?

Actually, we are talking about posix. I know POSIX doesn't include brk -
it doesn't
to allow the OS (any OS) to define its own method of allocating memory.
That doesn't
eliminate the fact that malloc allocates memory for a process to use.

> > > Again, the kernel gives you names for objects, not room to store them.
> > > Sorry, that's what you asked for, and that's what you get. If you want
> > > room for your objects, use another system call.
> > ---
> > What you are describing isn't reliable. What system call do I use
> > to reserve the Virtual Memory of the new process space after a fork? I don't
> > want to pin it in RAM, I just want a guarantee that Virtual Memory has been allocated.
>
> If there's no enough memory to *run* the process, it will fail.
> Why should the system consider the worst (uncommon) case and allocate
> the whole address space?

It doesn't fail. the system killed itself.

Irrelevent - malloc is the defined API, and the API is defined to return
null on allocation
failure. It doesn't do that in a consistant manner.

> > > Memory is accounted. Your problem is that brk() does not give you any
> > > memory. If you want memory, use another system call.
> > ---
> > Just because brk doesn't currently do real memory management doesn't
> > mean it shouldn't -- that's the crux of this issue. malloc is defined to
> > allocate *memory* -- not address space. You are confusing *existing*
> > behavior with *desired* behavior.
>
> Sorry, you are late. The malloc() argument has already failed. Read
> my previuos postings. Really, I won't repeat my arguments. Just a hint:
> you are confusing *programs* with *processes*.

That WOULD have been proper - except that it doesn't always do that.

> > > The kernel can't foresee the future. And it's silly to waste megs of
> > > space "just in case".
> > ---
> > But the user can. If your program is coded according to a "design".
>
> Use your own mm. Plain old malloc() is not what you need.

malloc is the defined API. We are not developing a few thousand APIs,
one for each
application. That wastes too much time for nonportable code.

> > > Use another system call. Implement your own memory management.
> > > Or just show a single example of an application, *written following the docs*,
> > > that will fail.
> > ---
> > This is the point -- why use a different system call? Why not just
> > allow the end user to configure whether or not the Linux kernel does address management
> > or memory management? What is your problem with configurability?
>
> It should be done by the application. This is Linux programming, not
> system management.

The application cannot depend on the system to supply resources.

> >
> > ---
> > There was a recent article pointer posted on 'slashdot' -- which talked about
> > how Linux has a long way to go before it has the maturity and stability to run
> > large commercial apps. You attitude is the reason why. Saying "we have a toy that
>
> I'm not aware of any OS which "has the maturity and stability to run large
> commercial apps". And, BTW, no "large commercial apps" that deserve a
> mature and stable OS.
> Don't repeat commercials slogans, I won't listen.
> I like your postings in which you make some technical points against
> Linux (which I'm not saying it's perfect). You don't need to point me
> to "articles" written by someone who's heard of Linux a couple of months
> ago for the first time.
>
> Anyway, my opinion doesn't count. Most of the main "large commercial apps"
> vendors are moving to Linux. This is a fact. Either directly or not,
> they're driven by technical reasons, since the Linux kernel has no
> marketing dept.

Many of them are having to rewrite parts of linux to do so. You seem to
be stuck in
a single user environment. As long as you remain there you will not
recognize the need
for proper resource management.

> > runs this way, deal with it" won't fly if someone requires no-overcommit. You can
>
> Only buggy applications require no-overcommit.

Only buggy systems mandate overcommit. Besides- init was killed, (or
sendmail/syslog/some
random user/bash/...) Not the small program that used the last available
page.

>
> > argue that they are wrong until you are blue in the face, but they won't care -- they
> > will just run NT or Solaris. The idea is to run with a kernel that performs resource
>
> And they will pay that choice. This is NOT a commercial vendor.
> This is the kernel development list. (Not speaking for others) I don't
> care a bit if someone else, in order to support badly written applications,
> uses another OS. They will pay that choice. Someone sooner or later
> will do the same job at half the costs (and the price) using Linux.
> I've seen this happen. I've seen companies not listening to my advices
> (at times when there was no hype around Linux) and choosing another,
> zero-management, OS, and later employing (full time) an administrator
> just to get a decent uptime. But this is OT.
>
> > management. Linux doesn't, as you have pointed out. It simply manages virtual address
> > page tables -- not real memory. Some people want a system to actually manage *memory*
> > not slots in a page table.
>
> Never said that. Your deliberately FUDing me (and Linux). I've never said
> there's no mm in Linux. I've just said that brk() does not allocate
> memory. The Linux kernel knows perfectly where RAM goes.

So it is lying to the process ?

>
> > > brk() is the normal API. And it works as expected.
> > ---
> > Not if I come from a system that manages memory.
>
> You don't like your application shows its bugs when run on a modern OS?
> Should I care?

A correctly functioning POSIX compliant program should work the same on
Linux as it
does on Solaris/Irix/UNICOS... Unfortunately they don't. Nothing wrong
with the program.
the system keeps killing random or sometimes the wrong programs. The OOM
killer is only
a band-aid to hide the fact that the kernel has problems in a multiuser
environment.

> > > I simply think it's wrong. "Proper functionality" *is* followed.
> > > You've got the docs. Read them, and write your programs after them.
> > ---
> > Just because a bug is documented still doesn't mean it is not a bug. Oh yeah,
> > it's not a bug...it's a *feature*. A proper feature operates under the principle
> > of "least surprise". Most users will expect malloc to allocate a real, usable
>
> Read the docs, and you have no surprise.
>
> > pointer. If the pointer is not usable, they will consider it broken. The fact
>
> Most DOS users expect for(i=0; i<10000000L; i++); is a legal way to get
> a delay in their interactive application. Is a perfectly legal ANSI-C
> piece of code, a definitely bad UNIX programming. I don't care what
> users expect. I read the docs, I know what brk() does.
>
> > that some arcane design describes how this really isn't a bug is irrelevant.
> > The user did not *ask* for such behavior. Most likely they want the behavior of
>
> *I* ask for such a behaviour.
>
> > getting back a *usable* pointer. Please -- don't try to argue that users really
> > don't care if malloc returns a bogus pointer. It won't fly.
>
> Never said that.

Sure you did. "malloc doesn't allocate memory..."

-- 
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@cats-chateau.net

Any opinions expressed are solely my own.

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Apr 07 2000 - 21:00:11 EST