Re: Is kernel optimized with dead store removal?

[Mikael Pettersson]

Roel Kluin writes:
 > According to http://cwe.mitre.org/data/slices/2000.html#14 due to optimization
 > A call to memset() can be removed as a dead store when the buffer is not used
 > after its value is overwritten. Does this optimization also occur during
 > compilation of the Linux kernel? Then I think I may have found some
 > vulnerabilities. One is sha1_update() where memset(temp, 0, sizeof(temp)); may
 > be removed.

Any such dead store removal is up to the compiler and the lifetime
of the object being clobbered. For 'auto' objects the optimization
is certainly likely.

This is only a problem if the memory (a thread stack, say) is recycled
and leaked uninitialized to user-space, but such bugs are squashed
fairly quickly upon discovery.

(checking gcc-4.4.3)
It seems that memset((volatile void*)&some_local_var, 0, sizeof(...))
just provokes a warning about the invalid type of memset()'s first
parameter, but then still optimizes the operation away.

You might need to call an out-of-line helper function for this to work.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/