[PATCH] ipc/sem.c: prevent ENOMEM in semop() w/ SEM_UNDO flag

From: Seiichi Ikarashi
Date: Fri Aug 03 2012 - 08:49:00 EST


semop() with SEM_UNDO sem_flg can result in ENOMEM even after
succeeding semget() with large nsems. This is because
semop() uses kzalloc() via find_alloc_undo() though
semget() uses vmalloc() via ipc_rcu_alloc().
This patch makes semop() be able to use vmalloc() via ipc_alloc().

Signed-off-by: Seiichi Ikarashi <s.ikarashi@xxxxxxxxxxxxxx>

--- a/ipc/sem.c 2012-08-03 16:52:01.000000000 +0900
+++ b/ipc/sem.c 2012-08-03 20:40:57.000000000 +0900
@@ -1258,11 +1258,12 @@ static struct sem_undo *find_alloc_undo(
sem_getref_and_unlock(sma);

/* step 2: allocate new undo structure */
- new = kzalloc(sizeof(struct sem_undo) + sizeof(short)*nsems, GFP_KERNEL);
+ new = ipc_alloc(sizeof(struct sem_undo) + sizeof(short)*nsems, GFP_KERNEL);
if (!new) {
sem_putref(sma);
return ERR_PTR(-ENOMEM);
}
+ memset(new, 0, sizeof(struct sem_undo) + sizeof(short)*nsems);

/* step 3: Acquire the lock on semaphore array */
sem_lock_and_putref(sma);
@@ -1348,7 +1349,7 @@ SYSCALL_DEFINE4(semtimedop, int, semid,
if (nsops > ns->sc_semopm)
return -E2BIG;
if(nsops > SEMOPM_FAST) {
- sops = kmalloc(sizeof(*sops)*nsops,GFP_KERNEL);
+ sops = ipc_alloc(sizeof(*sops)*nsops,GFP_KERNEL);
if(sops==NULL)
return -ENOMEM;
}
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/