mmap() weirdness in 2.2.11

Dick Streefland (dick_streefland@tasking.com)
Mon, 23 Aug 1999 17:28:59 +0200

After upgrading a number of machines to 2.2.11, we experienced
problems on three machines. The symptom is that the X server dies with
a segmentation fault on startup. The three machines that are affected
have different configurations, but all have 128 Mb RAM. The other
machines have less memory, so it looks like the problem is related to
the memory size. The problem was introduced in version 2.2.11, as
version 2.2.10 works OK. I can also reproduce it simply by starting
"acroread" on a system with 128 Mb memory.

The problem seems to be caused by an overlap between the memory
returned by mmap() and brk()/sbrk(). From the output of "strace
acroread":

...
mmap(0, 823296, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x8256000
mmap(0x8256000, 591973, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 4, 0) = 0x8256000
...
mprotect(0x8256000, 591973, PROT_READ|PROT_WRITE|PROT_EXEC) = 0
...
mprotect(0x8256000, 591973, PROT_READ|PROT_EXEC) = 0
...
brk(0x82565e4) = 0x825594c
brk(0x8255a24) = 0x8255a24
brk(0x8256000) = 0x8256000
brk(0x8257000) = 0x8256000
brk(0x8257000) = 0x8256000
write(2, "Unable to initialize AGM.\n", 26) = 26
write(2, "Unable to initialize application"..., 34) = 34
_exit(1) = ?

As you can see, brk() runs into the memory area of the mapping. With
the 2.2.10 kernel, or on a system with less that 128 Mb memory, mmap()
returns addresses around 0x40000000. BTW: the 2.2.11 kernel was
compiled with "CONFIG_1GB=y". It looks like the following change is
responsible for the problem:

--- v2.2.10/linux/mm/mmap.c Tue May 4 15:44:38 1999
+++ linux/mm/mmap.c Mon Aug 9 12:05:02 1999
@@ -359,8 +359,11 @@

if (len > TASK_SIZE)
return 0;
- if (!addr)
- addr = TASK_UNMAPPED_BASE;
+ if (!addr) {
+ addr = current->rlim[RLIMIT_DATA].rlim_cur;
+ if (addr >= RLIM_INFINITY)
+ addr = TASK_UNMAPPED_BASE;
+ }
addr = PAGE_ALIGN(addr);

for (vmm = find_vma(current->mm, addr); ; vmm = vmm->vm_next) {

However, I don't understand why this is releated to the physical memory
size.

-- 
Dick Streefland                      ////            TASKING Software BV
dick_streefland@tasking.com         (@ @)         http://www.tasking.com
--------------------------------oOO--(_)--OOo---------------------------


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/