I had a problem these days with one of our servers which is running kernel
2.0.33. A kernel oops occurred:
Jan 15 13:03:35 pop3 kernel: general protection: 0000
Jan 15 13:03:35 pop3 kernel: CPU: 0
Jan 15 13:03:35 pop3 kernel: EIP: 0010:[do_select+748/832]
Jan 15 13:03:35 pop3 kernel: EFLAGS: 00010002
Jan 15 13:03:35 pop3 kernel: eax: 74206567 ebx: 00b98024 ecx: 01d90248 edx: 74206567
Jan 15 13:03:35 pop3 kernel: esi: 00000207 edi: 01d70ed4 ebp: 0008d198 esp: 01d70e68
Jan 15 13:03:35 pop3 kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018
Jan 15 13:03:35 pop3 kernel: Process inetd (pid: 88, process nr: 14, stackpage=01d70000)
Jan 15 13:03:35 pop3 kernel: Stack: ffffffe9 01d70ef8 00000000 0000000a 00000000 00000001 00000001 01d9e100
Jan 15 13:03:35 pop3 kernel: 00b98000 00000000 00000001 00000003 00b98000 0012cb0e 0000000a 01d70f54
Jan 15 13:03:35 pop3 kernel: 01d70f14 01d70ed4 01d70f74 01d70f34 01d70ef4 bffffd38 0000000a bffffd00
Jan 15 13:03:35 pop3 kernel: Call Trace: [sys_select+670/988] [copy_page_range+405/912] [do_wp_page+0/668] [_getitimer+177/188] [_setitimer+108/268] [old_select+55/68] [system_call+85/128]
Jan 15 13:03:35 pop3 kernel: Code: 8b 42 04 39 d8 74 05 89 c2 eb f5 90 89 4a 04
56 9d 8b 4c 24
It seems that the process inetd caused this kernel fault. After that a very
strange thing happened: the talk wouldn't work any more(I did restart the
inetd daemon). In fact I restarted it while I was trying to see what's
happening with talk. First I thought that this is a configuration problem.
After looking more carefully into the system messages I noticed a long
thread of kernel oopses(the one above is the first one). Each of the
following oopses was caused by every "talk" process that I launched.
Another one which _I_ think is significant is:
Jan 15 14:00:47 pop3 kernel: Process talk (pid: 22520, process nr: 21, stackpage=01d18000)
Jan 15 14:00:47 pop3 kernel: Stack: 01d71c0c 00095898 01d71c0c 1166e2c1 11660206 00153264 01d9e34c 01d71c0c
Jan 15 14:00:47 pop3 kernel: 0014e9de 01d71c0c 0000005c 001cee50 00095818 00000000 01d18f80 00000000
Jan 15 14:00:47 pop3 kernel: 01d71c0c 005ce2c1 57055818 00000001 0014120f 00095898 001d8924 00000000
Jan 15 14:00:47 pop3 kernel: Call Trace: [def_callback2+24/44] [udp_rcv+982/1004] [ip_rcv+1167/1524] [net_bh+288/320] [do_bottom_half+59/92] [handle_bottom_half+11/32]
Jan 15 14:00:47 pop3 kernel: Code: 8b 0a 83 f9 01 75 69 9c 8f 45 fc fa c7 02 00
00 00 00 83 7a
Jan 15 14:00:47 pop3 kernel: Aiee, killing interrupt handler
Jan 15 15:16:01 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:01 pop3 kernel: q = 01d9e34c
Jan 15 15:16:01 pop3 kernel: *q = 00b98024
Jan 15 15:16:05 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:05 pop3 kernel: q = 01d9e34c
Jan 15 15:16:05 pop3 kernel: *q = 00b98024
Jan 15 15:16:09 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:09 pop3 kernel: q = 01d9e34c
Jan 15 15:16:09 pop3 kernel: *q = 00b98024
Jan 15 15:16:13 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:13 pop3 kernel: q = 01d9e34c
Jan 15 15:16:13 pop3 kernel: *q = 00b98024
Jan 15 15:16:18 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:18 pop3 kernel: q = 01d9e34c
Jan 15 15:16:18 pop3 kernel: *q = 00b98024
Jan 15 15:16:22 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:22 pop3 kernel: q = 01d9e34c
Jan 15 15:16:22 pop3 kernel: *q = 00b98024
Jan 15 15:16:26 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:26 pop3 kernel: q = 01d9e34c
Jan 15 15:16:26 pop3 kernel: *q = 00b98024
Jan 15 15:16:30 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:30 pop3 kernel: q = 01d9e34c
Jan 15 15:16:30 pop3 kernel: *q = 00b98024
Jan 15 15:16:34 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:34 pop3 kernel: q = 01d9e34c
Jan 15 15:16:34 pop3 kernel: *q = 00b98024
Jan 15 15:16:38 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:38 pop3 kernel: q = 01d9e34c
Jan 15 15:16:38 pop3 kernel: *q = 00b98024
Jan 15 15:16:41 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:41 pop3 kernel: q = 01d9e34c
Jan 15 15:16:41 pop3 kernel: *q = 00b98024
Jan 15 15:16:45 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:45 pop3 kernel: q = 01d9e34c
Jan 15 15:16:45 pop3 kernel: *q = 00b98024
Jan 15 15:16:49 pop3 kernel: wait_queue is bad (eip = 00153264)
Jan 15 15:16:49 pop3 kernel: q = 01d9e34c
Jan 15 15:16:49 pop3 kernel: *q = 00b98024
After that I killed the "inetd" process but I noticed that I still have
something listening on port 113(auth). A had a lots of tcp connections that
were in the "CLOSE" status and few in "ESTABLISHED" but on port 113. I am
sure that there was no process to listen because after that I decided to
restart the machine and I killed all processes(even agetty) and I still
could "telnet" to localhost on auth port.
Can someone tell me what happened?
Issue no 2(which is not related with kernel):
I do get an error in syslog:
Jan 17 20:54:56 pop3 syslog: main: accept. errno = 22
Any ideeas where it is coming from and why?
My machine is Pentium Intell 133Mhz, 32Mb, 2 network cards (Ne2000
Compatible) and nothing else.
The kernel config is:
#
# Automatically generated by make menuconfig: don't edit
#
#
# Code maturity level options
#
# CONFIG_EXPERIMENTAL is not set
#
# Loadable module support
#
CONFIG_MODULES=y
# CONFIG_MODVERSIONS is not set
# CONFIG_KERNELD is not set
#
# General setup
#
# CONFIG_MATH_EMULATION is not set
CONFIG_NET=y
# CONFIG_MAX_16M is not set
CONFIG_PCI=y
CONFIG_SYSVIPC=y
CONFIG_BINFMT_AOUT=y
CONFIG_BINFMT_ELF=y
CONFIG_KERNEL_ELF=y
CONFIG_M586=y
#
# Floppy, IDE, and other block devices
#
CONFIG_BLK_DEV_FD=y
CONFIG_BLK_DEV_IDE=y
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_BLK_DEV_IDE_PCMCIA is not set
# CONFIG_BLK_DEV_CMD640 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_TRITON=y
# CONFIG_IDE_CHIPSETS is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_MD is not set
CONFIG_BLK_DEV_RAM=y
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_BLK_DEV_XD is not set
# CONFIG_BLK_DEV_HD is not set
#
# Networking options
#
CONFIG_FIREWALL=y
CONFIG_NET_ALIAS=y
CONFIG_INET=y
CONFIG_IP_FORWARD=y
CONFIG_IP_MULTICAST=y
CONFIG_SYN_COOKIES=y
CONFIG_IP_FIREWALL=y
CONFIG_IP_FIREWALL_VERBOSE=y
CONFIG_IP_MASQUERADE=y
CONFIG_IP_MASQUERADE_ICMP=y
# CONFIG_IP_ALWAYS_DEFRAG is not set
CONFIG_IP_ACCT=y
# CONFIG_IP_ROUTER is not set
CONFIG_NET_IPIP=m
CONFIG_IP_ALIAS=y
# CONFIG_INET_PCTCP is not set
# CONFIG_INET_RARP is not set
# CONFIG_NO_PATH_MTU_DISCOVERY is not set
CONFIG_IP_NOSR=y
CONFIG_SKB_LARGE=y
CONFIG_IPX=m
# CONFIG_ATALK is not set
# CONFIG_AX25 is not set
# CONFIG_NETLINK is not set
#
# SCSI support
#
# CONFIG_SCSI is not set
#
# Network device support
#
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_EQUALIZER is not set
# CONFIG_PLIP is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_RADIO is not set
CONFIG_NET_ETHERNET=y
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
CONFIG_NET_ISA=y
# CONFIG_E2100 is not set
# CONFIG_DEPCA is not set
# CONFIG_EWRK3 is not set
# CONFIG_EEXPRESS is not set
# CONFIG_HPLAN_PLUS is not set
# CONFIG_HPLAN is not set
# CONFIG_HP100 is not set
CONFIG_NE2000=y
# CONFIG_SK_G16 is not set
# CONFIG_NET_EISA is not set
# CONFIG_NET_POCKET is not set
# CONFIG_TR is not set
# CONFIG_FDDI is not set
# CONFIG_ARCNET is not set
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# CD-ROM drivers (not for SCSI or IDE/ATAPI drives)
#
# CONFIG_CD_NO_IDESCSI is not set
#
# Filesystems
#
# CONFIG_QUOTA is not set
CONFIG_MINIX_FS=y
# CONFIG_EXT_FS is not set
CONFIG_EXT2_FS=y
# CONFIG_XIA_FS is not set
CONFIG_FAT_FS=y
# CONFIG_MSDOS_FS is not set
CONFIG_VFAT_FS=y
# CONFIG_UMSDOS_FS is not set
CONFIG_PROC_FS=y
CONFIG_NFS_FS=y
# CONFIG_ROOT_NFS is not set
CONFIG_SMB_FS=y
CONFIG_SMB_WIN95=y
CONFIG_NCP_FS=m
CONFIG_ISO9660_FS=y
# CONFIG_HPFS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
#
# Character devices
#
CONFIG_SERIAL=y
# CONFIG_DIGI is not set
# CONFIG_CYCLADES is not set
# CONFIG_STALDRV is not set
# CONFIG_RISCOM8 is not set
# CONFIG_PRINTER is not set
# CONFIG_SPECIALIX is not set
# CONFIG_MOUSE is not set
# CONFIG_UMISC is not set
# CONFIG_QIC02_TAPE is not set
# CONFIG_FTAPE is not set
CONFIG_APM=y
# CONFIG_APM_IGNORE_USER_SUSPEND is not set
# CONFIG_APM_DO_ENABLE is not set
# CONFIG_APM_CPU_IDLE is not set
# CONFIG_APM_DISPLAY_BLANK is not set
# CONFIG_APM_POWER_OFF is not set
# CONFIG_WATCHDOG is not set
# CONFIG_RTC is not set
#
# Sound
#
# CONFIG_SOUND is not set
#
# Kernel hacking
#
# CONFIG_PROFILE is not set
No modules were loaded!
LoadAverge was small. I run sendmail,ssh,httpd(apache 1.2).
-- Mircea Damian System Administrator dmircea@roedu.net, dmircea@lbi.ro, dmircea@kappa.ro MD65-RIPE, MD2225, MD1-6BONE Phone: +40-1-7719245