If kernel code accesses any page that is swapped out, the affected code
has to sleep while the page is swapped in. Some sections of the kernel
assume that the executing code cannot sleep. Break that assumption and
you get all sorts of nasty race conditions.
Many of the "cannot sleep" assumptions are done to avoid data specific
serialization. Along the lines of "I'm running, I own this data, no
other code can change it right now so I can do what I like to it".
With a fine grained kernel locking system, it may be possible to look
at swapping kernel pages but don't hold your breath, it is a
non-trivial exercise.