Boot no more

[KaiBo]

Some Plug of mine seems to have said goodbye for now. To make things worse it is in overseas, so I can hardly take a look at it. However, I managed to get an Image from the Plugs SD-Card, so I have Log-Files at hand.

First, the symptoms. I was building Kernels to get some PVR-Stuff up and running on the Plug. Everything went fine - and I only touched V4L-related things. However, a couple days later I could not log into the Plug anymore. Turned out that it was not available from its local net either. A reset did not help either. So, what does the log say about this?

Oct  2 12:33:33 zakhara kernel: ------------[ cut here ]------------
Oct  2 12:33:33 zakhara kernel: WARNING: at arch/arm/kernel/process.c:171 cpu_idle+0x80/0xb0()
Oct  2 12:33:33 zakhara kernel: Modules linked in:
Oct  2 12:33:33 zakhara kernel: Backtrace:
Oct  2 12:33:33 zakhara kernel: [<c002c024>] (dump_backtrace+0x0/0x110) from [<c03a7da0>] (dump_stack+0x18/0x1c)
Oct  2 12:33:33 zakhara kernel: r7:00000000 r6:c00299b8 r5:c0455294 r4:000000ab
Oct  2 12:33:33 zakhara kernel: [<c03a7d88>] (dump_stack+0x0/0x1c) from [<c003b438>] (warn_slowpath_common+0x50/0x80)
Oct  2 12:33:33 zakhara kernel: [<c003b3e8>] (warn_slowpath_common+0x0/0x80) from [<c003b480>] (warn_slowpath_null+0x18/0x1c)
Oct  2 12:33:33 zakhara kernel: r7:c04e2370 r6:c052076c r5:c04e237c r4:c04de000
Oct  2 12:33:33 zakhara kernel: [<c003b468>] (warn_slowpath_null+0x0/0x1c) from [<c00299b8>] (cpu_idle+0x80/0xb0)
Oct  2 12:33:33 zakhara kernel: [<c0029938>] (cpu_idle+0x0/0xb0) from [<c039f83c>] (rest_init+0x58/0x6c)
Oct  2 12:33:33 zakhara kernel: r6:c0023c98 r5:c0520510 r4:c05c7400
Oct  2 12:33:33 zakhara kernel: [<c039f7e4>] (rest_init+0x0/0x6c) from [<c00089a4>] (start_kernel+0x21c/0x270)
Oct  2 12:33:33 zakhara kernel: [<c0008788>] (start_kernel+0x0/0x270) from [<00008034>] (0x8034)
Oct  2 12:33:33 zakhara kernel: r6:c002409c r5:c0520798 r4:00053175
Oct  2 12:33:33 zakhara kernel: ---[ end trace e93d0df578db3594 ]---

This message appears like every other hour. It is always the exact same message that is being dumped. And from what I can see there is no other activity on the System (say: booting up into a defined state). Of course, services like SSHd are not available.

Anybody seen something like that? I am afraid that the Kernel-Portion of the Plug got hit by a bad-block (which means I would have to have the Plug shipped over to the US to take care of that).

[restamp]

The kernel errors may be a red herring.  See: http://plugcomputer.org/plugforum/index.php?topic=810.msg5057#msg5057

Good luck!

[KaiBo]

Kind of wondering why I did not find that when consulting the Forums Search-function. Anyway...

Thanks for pointing that out. As I did not upgrade my GIT-Sources in between my Kernel Builds (and the system was running stable before), this would be related to the most recent changes in my Kernel. I guess... However, that does not seem to fit:

Code:

/mnt# diff usr/src/linux/.config usr/src/linux/.config.old
4c4
< # Tue Sep 29 15:25:03 2009
---
> # Tue Sep 29 15:09:54 2009
1039c1039
< CONFIG_MEDIA_TUNER_XC5000=y
---
> CONFIG_MEDIA_TUNER_XC5000=m
1096c1096
< CONFIG_VIDEO_CX231XX_DVB=y
---
> # CONFIG_VIDEO_CX231XX_DVB is not set

However, my symptoms to not fit anymore anyway. I had an fsck done to the SD-Card which holds my OS and after fixing several issues it now boots again. Now I am stuck during login as it just "hangs" after entering my password. At this point I am really curious about what is going on there.

My next step will be to install the Kernel from the bogus Plug on the Plug I have here. Maybe I can recreate the problem and figure out a solution (f.ex. see if that Kernel-Patch might help).

[KaiBo]

Even though I did not really get to install the Kernel to my other Plug, I did find that login into the bogus Plug always stalls at the same point. Using debug-output I discovered that the Login itself works quite well (even though horribly slow). The process seems to hang when sh is about to be invoked:

Code:

debug2: we sent a password packet, wait for reply
debug1: Authentication succeeded (password).
debug1: channel 0: new [client-session]
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug2: callback start
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug2: channel 0: request shell confirm 1
debug2: fd 3 setting TCP_NODELAY
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768

Now, I am not an SSH-Expert. But from what I have found on the Web this seems to be related to a read-command for the filesystem (invocation of sh as mentioned above would be my guess). I am beginning to think that the SD card itself might be broken. Any clues from the experts on this?