Re: [ANNOUNCE] 3.0-rc7-rt0

From: Geunsik Lim
Date: Wed Jul 20 2011 - 05:49:01 EST


Wow.. Its' good news for RT guys that are trying to research and develop
soft-realtime system based on latest Linux version. :)
Thank you for a lot of efforts for Real-time Linux for a long time.

On Wed, Jul 20, 2011 at 9:37 AM, Thomas Gleixner <tglx@xxxxxxxxxxxxx> wrote:
> Dear RT Folks,
>
> I'm pleased to announce the first drop of the 3.0-rc7 based RT
> patch.
>
> It's been quite a while since 2.6.33-rt, but I went through a very
> painful experience while trying to get a 2.6.38-rt stabilized. The
> beast insisted on destroying filesystems with reproduction times
> measured in days and the total refusal to reveal at least a
> minimalistic hint to debug the root cause. Staring into completely
> useless traces for months is not a very pleasant pastime.
>
> That's the very first problem in the RT history which I gave up on.
>
> [The truth: Linus avoiding the final 2.6.42 release made all my
> Âultimate plans go down the drain ... ]
>
> Though while trying to analyse the problem I had plenty of time to
> twist my brain around the existing RT approach and its shortcomings.
>
> The main issue which RT is fighting with is the ever growing per cpu
> variable usage and the assumptions which are built around it. The
> existing RT approach to work around this with PER_CPU_LOCKED
> constructs and hand the CPU number around simply does not work anymore
> because the number of sites which need to be patched is way too large
> and the resulting mess in the code is neither acceptable nor
> maintainable.
>
> After lenghty and fruitful discussions with Peter Zijlstra - thanks a
> lot Peter! - we finally agreed on trying a totally different approach
> to tackle these issues: disabling migration over spinlock and get_cpu
> sections. This had been discussed before, but nobody ever considered
> to sit down and make it work.
>
> This keeps the semantics which are expected by the per cpu users,
> while keeping the regions preemptible. As a side effect, it allows us
> to run softirq handlers directly from irq threads on local_bh_enable
> which was a long desired feature to lower the performance impact of
> RT.
>
> Changing this required a major refactoring of the RT patch queue,
> which took some time as I had to go through every single patch, fold
> fixes back into the right places and sort them into various categories:
>
> Â- Mainline ready (raw lock annotations, infrastructure patches, code
> Â restructuring...)
>
> Â- Preparatory (_rt()/_nort() variants of preempt_*(), local_irq_*(),
> Â BUG*(), WARN*() and the annotations in various places)
>
> Â- Base patches (Reworking the slab/page_alloc code, bit_spinlock
> Â replacements, migrate disable infrastructure ...)
>
> Â- Full RT patches (sleeping spinlocks and the resulting fixups here
> Â and there)
>
> In course of that exercise I weeded out a lot of historically grown
> hackery and dropped stuff which was not essential for getting it up
> and running. Thanks to Carsten for reintegrating the tracer addons
> which he's using for the OSADL test farm:
>
> Âhttps://www.osadl.org/?id=1042
>
> I probably have missed a few bits and pieces, but the overall outcome
> is stable and survived testing on various systems. The latency
> behaviour with cyclictest is on par with 33-rt at least on x86_64/32.
>
> The overall patch size has shrunk significantly and the readability
> (except for the missing changelogs in various patches) is at an
> acceptable level.
>
> If you download the quilt tarball, you'll find various sections:
>
> - upstream fixes: Stuff broken upstream which we managed to trip
> Âover. This section contains real weird stuff from simple fixes, over
> Âmainline code which claims to contain (complete bogus) RT support up
> Âto an archaeologic bug in the floppy driver code.
>
> Â8 patches (size 8892)
> Â7 files changed, 59 insertions(+), 51 deletions(-)
>
> - upstream submitted: Stuff which is on LKML already and needs some
> Âfollow up.
>
> Â4 patches (size 9741)
> Â4 files changed, 81 insertions(+), 119 deletions(-)
>
> - upstream ready: Stuff which needs a bit polishing and upstream
> Âsubmission
>
> Â79 patches (size 232566)
> Â192 files changed, 1204 insertions(+), 1097 deletions(-)
>
> - upstream needs work: Stuff which should go upstream, but needs some
> Âor lots of care.
>
> Â7 patches (size 164120)
> Â49 files changed, 3292 insertions(+), 253 deletions(-)
>
> - the real rt stuff:
>
> Â125 patches (size 280665)
> Â162 files changed, 4327 insertions(+), 592 deletions(-)
>
> The overall patch is now:
> Â223 patches (size 680054)
> Â374 files changed, 8950 insertions(+), 2099 deletions(-)
>
> Compared that to 2.6.33-rt:
> Â462 patches (size 1396505)
> Â690 files changed, 15994 insertions(+), 5123 deletions(-)
>
> That's a significant reduction in size and impact. Some of it is due
> to the new approach, but we also got quite a lot of the infrastructure
> patches upstream in the last few kernel releases. Thanks to all folks
> who have helped to get that done, especially to Peter Zijlstra for
> getting the preemptible mmu gather problem and lots of the scheduler
> issues which we discovered in RT over time sorted out!!!
>
> What's new in 3.0-rt ?
>
> Â- No more split soft interrupt threads. We need to analyze whether
> Â this is a good decision.
>
> Â- softirq handling from the end of interrupt threads and on all
> Â thread sites where a nested local_bh disabled section ends
>
> Â- SPARSE interrupts and IOMMU interrupt remapping work now
>
> Â- Split config option CONFIG_PREEMPT_RT into CONFIG_PREEMPT_RT_BASE
> Â and CONFIG_PREEMPT_RT_FULL. RT_BASE covers some of the more complex
> Â changes (e.g. mm/* where we substitute interrupt disabled sections
> Â with per cpu locks and the bit_spinlock to spinlock conversion).
> Â RT_BASE allows us to test and verify these changes independently of
> Â the big RT_FULL modifications. That's mainly a debugability and
> Â maintainability issue.
>
> What's the state:
>
> Â We've done quite some testing on x86 32/64 bit and basic tests on
> Â some ARM/MIPS/POWERPC platforms. Thank God, no file system eating so
> Â far :)
>
> Â Given the fact that it is a major rewrite it's amazinlgy stable and
> Â I consider it to be the best -rt1 release we ever had. That doesn't
> Â mean that there are no bugs, since it has not had the proper test
> Â coverage yet.
>
> Â Thanks to Carsten, Clark and Peter for all the help to get this far!
>
> Want to help?
>
> Â Many people offered help in the past and I had to turn them down so
> Â far as refactoring that stuff really is not a task which can be
> Â shared easily. Though now is the point where I can use all the help
> Â you promised to provide.
>
> Â What's needed?
>
> Â - Testing, testing, testing ... you know the drill (good bug
> Â Â reports are 98% of the solution)
>
> Â - Compare and analyze the performance/troughput impact of the new
> Â Â approach with 33-rt
>
> Â - Help mainlining the "upstream ready section"
>
> Â Â That means reviewing the patches, cleaning them up, fixing the
> Â Â changelogs, submitting them through the proper channels ...
>
> Â Â Please do not blindly pick any of these patches and submit them
> Â Â to mailing lists w/o doing the above. Also please coordinate on
> Â Â the #linux-rt IRC channel on oftc.net so redundant and
> Â Â conflicting work can be avoided
>
> Â - Help getting the "upstream needs work" section into shape
>
> Â Â All of these patches need a close look and (especially the
> Â Â hwlatency detector) major cleanups. Please coordinate with the
> Â Â patch authors and lookout for previous discussions of some of
> Â Â those on LKML.
>
> Â - Tend to the FIXME annotations in the RT stuff section
>
> Â Â I have annotated some places with /* FIXME ... comments. These
> Â Â sections are not for the faint hearted and need some serious
> Â Â review and thought.
>
> Â - Help with the RCU modifications
>
> Â Â That's an easy one. We have a volunteer signed up for this
> Â Â involuntarily already. Thanks Paul!
>
> Â - Twist your brain around the schedulability impact of the
> Â Â migrate_disable() approach.
>
> Â Â A really interesting research topic for our friends from the
> Â Â academic universe. Relevant and conclusive (even short notice)
> Â Â papers and/or talks on that topic have a reserved slot in the
> Â Â Kernel developers track at the Realtime Linux Workshop in Prague
> Â Â in October this year.
>
> Enough marketing, here comes the real stuff.
>
> ÂPatch against 3.0-rc7 can be found here:
>
> Â Âhttp://www.kernel.org/pub/linux/kernel/projects/rt/patch-3.0-rc7-rt0.patch.bz2
>
> ÂThe split quilt queue is available at:
>
> Â Âhttp://www.kernel.org/pub/linux/kernel/projects/rt/patches-3.0-rc7-rt0.tar.gz
>
> There is no git tree for now.
>
> I'm not yet convinced that moving RT to git was a good idea as quilt
> allows me to move stuff around in a way more flexible manner. So for
> now no git version until someone comes up with a brilliant idea which
> allows me to keep my workflow sane (do not even try to suggest stgit &
> co!).
In my opinion, many developers will want to use -rt tree based on git
like previous -tip tree for rt patch. :)
For example, http://git.kernel.org/?p=linux/kernel/git/tip/linux-2.6-tip.git
>
> That said, have fun and make sure that you have the fire extinguisher
> ready when you start using this!
>
> Thanks,
>
> Â Â Â Âtglx
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at Âhttp://vger.kernel.org/majordomo-info.html
> Please read the FAQ at Âhttp://www.tux.org/lkml/
>



--
Regards,
Geunsik Lim ( Samsung Electronics )
Blog : http://blog.naver.com/invain/
e-Mail: geunsik.lim@xxxxxxxxxxx
    Â leemgs@xxxxxxxxx , leemgs1@xxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/