PTY patch

Albert Cahalan (albert@ccs.neu.edu)
Sat, 28 Sep 1996 15:51:01 -0400 (EDT)


Earlier I posted that someone fixed the PTY problems and posted
a patch to comp.os.linux.development.kernel. Since the patch did
not make it to the mailing list and there is no 2.0.22 with the
fix yet, I went and got the patch from AltaVista.

> 2.0.X PTY patch (was: Bug in PTY handling)
>
> From thorsten@actis.de (Thorsten Kuehnemann)
> Date 26 Sep 1996 18:17:58 +0200
> Newsgroups comp.os.linux.development.system
>
> I wrote:
> >some people have found anomalies in pty-handling. Please test this:
>
> >- start a new "xterm"
> >- in that xterm, start a "xclock &" (in background)
> >- exit that xterm
> >- start "script"
>
> >If "script" exits immediately, you have that PTY bug too:
>
> >$ script
> >Script started, file is typescript
> >Script done, file is typescript
> >$
>
> After days of isolating the bug and learning about implementation specifics
> of PTYs in Linux and other operating systems i made (and tested) a patch
> for Linux-2.0.21.
>
>
> It's a bugfix, not a new feature!
>
>
> With that patch you can't open a pty-master if there is an old process
> always running at the slave side of the pty. This emulates the behaviour
> of HP-UX 10, Solaris 2.4, Sinix 5.42 and SCO OS5 and makes programs
> like screen, script, xconsole, old xterm and others work with Linux-2.0.X
> even under special conditions.
>
>
> The change is in the function pty_open():
>
>
>
> --- v2.0.21/linux/drivers/char/pty.c Wed Jul 17 14:10:03 1996
> +++ linux/drivers/char/pty.c Thu Sep 26 16:31:43 1996
> @@ -196,6 +196,10 @@
> pty = pty_state + line;
> tty->driver_data = pty;
>
> + if ((tty->driver.subtype == PTY_TYPE_MASTER) &&
> + (tty->link->count != 1) )
> + return -EBUSY;
> +
> if (!tmp_buf) {
> unsigned long page = get_free_page(GFP_KERNEL);
> if (!tmp_buf) {
>
>
>
>
>
> The funktion tty_open() calls init_dev() which increments tty->link->count
> just before tty_open() calls pty_open(). That is why i check the count
> against 1, not 0.
>
> It is important to do this check before clearing any CLOSED-Bits or
> TTY_THROTTling.
>
> The errno EBUSY describes (in my opinion) the state of that pty best.
>
>
> Please test this if you have the time to and mail me if
> it fails.
>
>
> I would like to thank William Burrow for taking my problem report serious
> and for taking the time to help me to understand and solve the problem.
>
> This is what i thought it is the normal case: The cooperative working
> on problems in Linux. But i learned that for problem reports from
> non-kernel-gurus like me and most other users this is the exception :-(
>
> But I thank all others who told me that they have PTY-problems in
> Linux-2.0.X, too. It was very important for me to know that the PTY-Bug
> was not special to my system or distribution.
>
> Thorsten
> --
> PGP fingerprint = 55 D7 E6 6A 08 D0 C8 F4 6B AC C2 B7 64 AD A0 F6
> My opinions are my own, except those I've stolen.