• Home
  • Help
  • Search
  • Login
  • Register
Pages: [1]
Author Topic: sheevaplug USB mount fails periodically (debian)  (Read 7135 times)
Zortrium
Newbie
*

Karma: 0
Posts: 31


View Profile
« on: July 20, 2010, 09:54:22 AM »

I've got my plug booting completely off an external USB drive (both root directory and kernel) running Debian Squeeze.  The problem is that occasionally, maybe once every two weeks, the USB mount seems to fail out of nowhere.  When this happens, the plug still respond to pings normally, but I can no longer SSH into it (I just get ssh_exchange_identification: Connection closed by remote host), and while Apache remains active, every request returns a 403 Forbidden, presumably because the plug can't access anything on the filesystem.  Whenever this happens, I reboot the plug by hand and everything comes back up (until the next time it happens).

This is annoying for two primary reasons: one, I have no idea why it's happening, and two, when it happens, I have no way to remotely reboot, since I can't get a shell.  Anyone have any ideas as to why it might be happening or what I could try to deal with it?
Logged

tylernt
Jr. Member
**

Karma: 2
Posts: 56


View Profile
« Reply #1 on: July 20, 2010, 12:02:37 PM »

What does /var/log/syslog and/or /var/log/messages say leading up to the failure? Can you leave the USB-serial port running all the time with logging enabled on the other end so you can see if the kernel console spews anything useful at the failure point?

To work around the issue, perhaps you can configure a small RAMDisk and run SSH from it (so it's available without the HD). (You may need to compile SSH as a static binary or use Dropbear so it doesn't go looking for libraries on the HD.) Then you could do something like

http://linux.derkeiler.com/Mailing-Lists/Kernel/2005-10/3263.html

to SSH in and force an immediate hard reboot.

If the problem usually takes more than a week to show up, you might consider adding a cron job to reboot it weekly as a pre-emptive measure.
Logged

Zortrium
Newbie
*

Karma: 0
Posts: 31


View Profile
« Reply #2 on: July 20, 2010, 04:21:12 PM »

I opened up the serial console (while the plug was still in the messed-up state) and got this:

[2061996.143493] EXT3-fs error (device sda2): ext3_find_entry: reading directory #1183320 offset 0
[2061996.152312] EXT3-fs error (device sda2): ext3_find_entry: reading directory #1183225 offset 0
[2061996.162646] Read-error on swap-device (8:0:44040)                                     
[2061996.177952] Kernel panic - not syncing: Attempted to kill init!
[2061996.184174] [<c002d0b8>] (unwind_backtrace+0x0/0xdc) from [<c02b3f20>] (panic+0x34/0x128)
[2061996.192571] [<c02b3f20>] (panic+0x34/0x128) from [<c004777c>] (do_exit+0x6c/0x64c)
[2061996.200372] [<c004777c>] (do_exit+0x6c/0x64c) from [<c0047de4>] (do_group_exit+0x88/0xbc)
[2061996.208788] [<c0047de4>] (do_group_exit+0x88/0xbc) from [<c0054d7c>] (get_signal_to_deliver+0x2f4/0x330)
[2061996.218508] [<c0054d7c>] (get_signal_to_deliver+0x2f4/0x330) from [<c0029590>] (do_notify_resume+0x70/0x604)
[2061996.228578] [<c0029590>] (do_notify_resume+0x70/0x604) from [<c0026ee8>] (work_pending+0x1c/0x20)

The system logs didn't look to have anything relevant once I rebooted the plug.
Logged

creeble
Newbie
*

Karma: 0
Posts: 20


View Profile
« Reply #3 on: September 09, 2010, 11:40:06 AM »

That looks like either an actual disk failure (fsck'd it lately?) or perhaps it's spinning down and won't spin back up.  Some USB drives (notably the Seagate 1TB externals) are set to automatically spin down, and Linux doesn't know about it.  There are workarounds using sdparm.

Perhaps using sdparm periodically to check disk status would be useful?

Logged

Zortrium
Newbie
*

Karma: 0
Posts: 31


View Profile
« Reply #4 on: September 09, 2010, 01:45:31 PM »

Yeah, I think the disk itself (or possibly the enclosure) is going south.  It started crapping out after a period of hours rather than days and at one point failed before it was even done rebooting.  It's sitting on my disk waiting for some trial and error troubleshooting at the moment.
Logged

UnaClocker
Full Member
***

Karma: 0
Posts: 131



View Profile WWW
« Reply #5 on: October 31, 2010, 10:27:58 PM »

I had a very similar problem, drove me nuts for months, luckily my root filesystem was on an SD card, my 1TB USB drive was just a media storage drive, which my Sheeva serves up from.. Finally, I got fed up with it crashing all the freaking time, and bought a 2TB drive in a new enclosure, figuring it must be my external enclosure being flaky. I removed the drive and put it in my desktop machine and immediately got a boatload of SMART warnings. The drive had FAR exceeded it's safe level of bad sectors.. The drive was just flat NFG.. It's a shame Linux didn't pick that up through the USB connection and tell me sooner.. Would have saved a lot of heartache. On the plus side, the new drive has energy saving features like automatic spindown. Being a media serving drive, that's a good thing in my book, little 2 second pause before a movie starts playing on the PS3 is fine with me. Wink
Logged

SheevaPlug - 8gb class 4 SDHC primary drive, 4tb 3.5" media drive, Debian Wheezy, nginx, Samba, Shorewall

cjm
Jr. Member
**

Karma: 6
Posts: 69


View Profile
« Reply #6 on: November 29, 2010, 01:27:15 PM »

Is the hard disk attached to a USB hub? I had numerous issues with cheap USB hubs which kept disconnecting themselves (and all attached devices) every few days or weeks. Right now I'm using a Logitec hub which, unfortunately, is no longer available but with this hub the plug is running for months without any problems whatsoever. The chipset in the hub is:

  0409:005a NEC Corp. HighSpeed Hub

Thanks,
--Christian
Logged

sid
Newbie
*

Karma: 0
Posts: 1


View Profile
« Reply #7 on: June 25, 2011, 03:45:42 AM »

Hi,

I have seen the same kind of issue on my sheeva setup. I have two hard disks connected through an USB hub. The first one works very well, and never seems to fail, whereas the second one periodically fails, once every two weeks approximatively, like you. It seems that this latter one has automatic spin down function enabled.

I don't have global issues, such as ssh connection, as the hard disk are only used for data.

What have you finally done to resolve your problem ?

Thank you,
sid.
Logged

Tom
Newbie
*

Karma: 0
Posts: 11


View Profile
« Reply #8 on: June 26, 2011, 03:28:20 AM »

Are your USB drives powered externally or by sheevaplugs power supply? I had the same issue when I changed my external powered 3.5" usb hdd to a 2.5" hdd (WD with reduced spinup power) which is powered by the sheevaplug only (so no extra power supply). I wanted to change to 2.5" to reduce my power costs but got an unstable system.

My workaround was to use a Y-cable for a second usb power source and conneted the second cable end to one of the free usb ports on my OpenRD client. No problems afterwards. My conclusion is that Sheevaplugs internal power supply isn't able to power the plug AND an external hdd even if it has reduced power consumption. Maybe a SSD drive would be ok.
Logged

Zortrium
Newbie
*

Karma: 0
Posts: 31


View Profile
« Reply #9 on: July 01, 2011, 12:35:00 PM »

FYI, all the hard drives I've plugged into my plug were on external power, so power from the USB bus was never the issue in my case.  I swapped out the hard drive into a new USB enclosure and that seemed to fix the problem -- don't think I've seen it in the past couple of months.
Logged

Pages: [1]
Print
Jump to: