Hi,
as I told you, I am preparing a test box outside my firewall and a
test case and I'm very willing to give you a login and every support I
can do. But then again I'm not a student and I have projects to work
on and money to earn so I do this preparation in my spare time and it
will not happen today. Maybe tomorrow (Monday is holiday in Germany)
(If anyone can spare a running config of Steel belted Radius SPE with
Netscape directory server, I would get even more free time sooner. ;-) )
If you want to take a stab onto it without waiting for me, here is what my
application does:
- use Linux 2.2.15pre17 with ReiserFS 3.5.19 (ugh, your .19 patch
reports itself as .18 in the log files. :-( )
I use RedHat 6.2 with this custom kernel as an addition. I prepared
the RPMS and SRPMS for you on ftp://ftp.hometree.net/pub/reiser/
- My machine is
- RedHat 6.2 (glibc 2.1.3)
- PostgreSQL 6.5.3 as delivered with RedHat 6.2
- custom compiled kernel (using egcs-1.1.2 release):
2.2.15pre17
- ide-20000325 driver (but no IDE hardware installed and modules
for IDE not loaded)
- reiserfs-3.5.19 driver
- Intel Pentium III Processor (600 MHz, 100 MHz FSB)
- 512 MB RAM
- Gigabyte GA6-BX7 board (intel 440BX based)
- Matrox G-200 Videoboard
- Adaptec 29160 SCSI hostadapter , driving an 9 GB IBM and a 120 GB
Arena HW-RAID
- 3COM 3c905B Network card
- Cyclades Cyclom YeP with 16 serial ports (DB-25 box)
- run ReiserFS on a SCSI disk off an Adaptec 29160 (System resided on
another disk on the same controller). I used a single partition on a
dedicated disk, /dev/sdb1 was the same size as the whole disk. The disk
in my test case is the above mentioned ARENA RAID but it presents itself
to the host as a single SCSI disk, not a RAID array.
- On RedHat, wipe the /var/lib/pgsql directory. Create a directory called
"pgsql" on the new partition. chown in postgres.postgres. link this
directory from /var/lib. If you now start the postgresql, it will
create the necessary housekeeping stuff on the new disk.
- Prepare a program which does the following things:
- create a new database inside postgresql
- create a table with two columns: one contains an int, the other an
Oid.
do a loop (1000-2000 times. Fixed number, same for each run):
- start a transaction
- create a large object of random size (mine are up to 8k)
write its content with a known, reproduceable pattern. You could use
the first some k of /boot/vmlinuz e.g.
- insert a row into the table above. Int column is the created size,
Oid column is the oid of the large object
- commit the transaction
- Prepare a program which opens the a database and checks the following
things
- number of rows in the table
- existence of the large object referenced by the Oid in the table
- size of the large objects in comparisation to the number in the
int column
- contents of the large objects in comparisation to the known file
- now write a script which starts 10-25 instances (depending on your disk
size and horsepower) of the first program. Let them run in parallel
until finishing. There should now be 10-25 databases with a table and
a number of large objects. The tables should all be equal sized and contain
rows with large object references and size integers.
- in a loop run the second program to validate the integrity of the database.
In a nutshell this is what my application does. This above has not yet
been coded as a test case. My application corrupts the database on
file system level on ReiserFS 24 out of 25 times. You may correctly
guess that finding this problem took me lots of time. Since I'm back
on ext2, I can run my application for ages without any problems.
I'm guessing (wild guess here), that PostgreSQL is very heavy on
mmap()ing files into memory. It is also a heavy shared memory
user. Maybe there are some unfortunate interactions here. It is (to my
knowledge) not a threaded application.
Hans, I'm _very_, _very_ interested into getting these things fixed. I
consider a journalling file system the second most important addition
to the Linux kernel (the first is a working, performant V3 NFS over
TCP) and ReiserFS ist definitely a step in the correct direction.
Fixing the bugs on 2.2.x is also important to me because 2.2.x IMHO
well be around for quite a while before general adoption of 2.4.x (I
still have a couple of 2.0.35 boxes... :-) )
Regards
Henning
On Sat, Jun 10, 2000 at 02:52:26PM -0700, Hans Reiser wrote:
> Rik van Riel wrote:
> >
> > On Sat, 10 Jun 2000, Hans Reiser wrote:
> >
> > > We would definitely like to investigate this. It sounds like
> > > postgresql depends on an assumption that any linux fs supports
> > > some ext2 only features, thus making it a postgresql bug, but we
> > > want to take a look.
> >
> > I wouldn't call O_SYNC ext2 specific ... ;)
> >
> > regards,
> >
> > Rik
> > --
> > The Internet is not a network of computers. It is a network
> > of people. That is its real strength.
> >
> > Wanna talk about the kernel? irc.openprojects.net / #kernelnewbies
> > http://www.conectiva.com/ http://www.surriel.com/
>
> Apologies for my weaseling, we will fix O_SYNC, and we will fix the postgresql
> bug whatever it is if he lets us login or if we can independently reproduce it.
>
> Hans
-- Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- hps@tanstaafl.de TANSTAAFL! Consulting - Unix, Internet, SecurityHutweide 15 Fon.: 09131 / 50654-0 "There ain't no such D-91054 Buckenhof Fax.: 09131 / 50654-20 thing as a free Linux"
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Jun 15 2000 - 21:00:22 EST