Dead SD Card after 3 weeks : caused by Sheevaplug or something else?

[PlugPBX]

So this is easy

Drop the Flashybrid script into rcS.d - BEFORE the ethernet interface starts up...

Code:

PlugPBX:/etc/rcS.d# ls
README      S05hwclockfirst.sh    S07udev-mtab      S11mountoverflowtmp  S15mountnfs.sh
S01mountkernfs.sh    S06checkroot.sh    S08checkfs.sh      S12networking   S16mountnfs-bootclean.sh
S02udev      S07hwclock.sh    S09ifupdown      S12procps   S17bootmisc.sh
S03mountdevsubfs.sh  S07ifupdown-clean    S09mountall.sh      S12x11-common   S17urandom
S04bootlogd      S07module-init-tools  S10mountall-bootclean.sh  S13portmap   S18stop-bootlogd-single
S05hostname.sh      S07mtab.sh    S11flashybrid      S14nfs-common

Notice the S11flashybrid set to ...well, 11.

Works perfect with the following mappings for the PlugPBX debian system (as an example)

/etc/flashybrid/ramstore

Code:

/etc
/var/cache/samba
/var/lib/samba
/var/backups
/var/lib/asterisk
/var/lib/postfix
/var/lib/dbus
/var/lib/logrotate
/var/lib/mysql
/var/webmin
/var/log
/var/run
/root
/var/spool
/var/mail

and

/etc/flashybrid/ramtmp

Code:

/tmp
/var/lock
/var/lib/dhcp3
/var/lib/php5
/var/lib/misc
/var/lib/ntp
/var/lib/urandom
/var/tmp

There you have it. A system running read only from SD card.

Want to sync whats changed back to SD card? Execute

Code:

fh-sync

Want to updated the system, alter any other files?

Code:

mountrw

All done and want to lock it back down to a read only system

Code:

mountro

Apt-get install flashybrid, and follow the steps above and adapt to services that need write file access at runtime.

Think of this as a 'router' experience, but at the flip of a switch its a fully maintainable server. Flip back and its locked down.

Need logs saved to storage? Setup a cron script to run fh-sync to your level of preference - you are VASTLY reducing writes to SD cards and instead of getting a year or two, it'll out last your lifespan.

Screw up something really bad? Don't reboot it, just unplug it and power back up. It'll boot from the last saved state from SD card. No messy filesystem fsck required. It'll tolerate crappy power as long as it's not syncing to disk at the time

Update: Almost forgot. Until you cleanly work with debians rc scripts (depends what kind you are using) updates / installations can upgrade these manually mucked about rc scripts, charging run level ordering - and its important WHEN flashyrbid is started. I've had the S11 changed to S20 installing ntp after the fact. So you are advised to do all this 'cleanly' - I'm reading up now on just how exactly one marks the ordering so debian will always honor the ordering I want.  I'm from the old school rc.d days it seems and got surprised by this new way things work in debian.

Other then this final snag, this works fantastic. Its the lazy mans embedded trick

[cjm]

Cool -- sounds like a completely feasible solution for root file systems on SD cards!

Regarding the start time: The init scripts in Debian have a section like this (taken from sendmail):

Code:

### BEGIN INIT INFO
# Provides:          sendmail
# Required-Start:    $remote_fs $network $syslog
# Required-Stop:     $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      1
# Short-Description: powerful, efficient, and scalable Mail Transport Agent
# Description:       Sendmail is an alternative Mail Transport Agent (MTA)
#                    for Debian. It is suitable for handling sophisticated
#                    mail configurations, although this means that its
#                    configuration can also be complex. Fortunately, simple
#                    thing can be done easily, and complex thing are possible,
#                    even if not easily understood ;)  Sendmail is the *ONLY*
#                    MTA with a Turing complete language to control *ALL*
#                    aspects of delivery!
### END INIT INFO

It should be possible to use the Default-Start and Required-Start sections to survive runs of update-rc.d. This should keep you safe until flashybrid is udpated. The next step would probably be to submit a defect with the Debian folks to have this done per default -- given all the problems when starting flashybrid later, doing so sounds like a bug to me...

[hausschuh]

I haven't read all of your posts but as far as I can see you have the problems I have thought of before.
I'm going to install Debian (Squeeze) on the Sheevaplug and it only supports installing it on SD (as far as I know).

So my concerns are about the limited read/write cycles of the SD card.
Is the OS booted into the RAM so while the Sheevaplug has power there won't be any (unnecessary) writing/reading cycles?
Does the (access)speed of the SD Card have influence on the overall system speed (like copying files over the network, unpacking rar files on the HDD plugged to the Sheevaplug)?
What about the apt-get update files beeing installed? Are they stored in RAM first before written to disk? I would like to install debian squeeze, but since it is under (heavy) development the files will change several times a week and would crash the SD in no time.
What is the size of Debian with some packages like openssl, libcurl, nscurses, rtorrent, libtorrent, proftpd? Is a 2GB SD card enough or should I head for at least 4GB?
Is there any advantage in the SDHC besides "High(er) Capacity" concerning the Sheevaplug?

[PlugPBX]

I haven't read all of your posts but as far as I can see you have the problems I have thought of before.
I'm going to install Debian (Squeeze) on the Sheevaplug and it only supports installing it on SD (as far as I know).


So my concerns are about the limited read/write cycles of the SD card.
Is the OS booted into the RAM so while the Sheevaplug has power there won't be any (unnecessary) writing/reading cycles?
Does the (access)speed of the SD Card have influence on the overall system speed (like copying files over the network, unpacking rar files on the HDD plugged to the Sheevaplug)?
What about the apt-get update files beeing installed? Are they stored in RAM first before written to disk? I would like to install debian squeeze, but since it is under (heavy) development the files will change several times a week and would crash the SD in no time.
What is the size of Debian with some packages like openssl, libcurl, nscurses, rtorrent, libtorrent, proftpd? Is a 2GB SD card enough or should I head for at least 4GB?
Is there any advantage in the SDHC besides "High(er) Capacity" concerning the Sheevaplug?

Last time I checked, yes install to SD media only easily via Squeeze installer...

There are articles on the PlugComputer.org wiki detailing how you can 'minimize' writes to SD cards.

I find a good performance / price point are Class 6 SD cards. I've seen class 10 cards (things scream) but its over kill.

Remember for most systems, once its booted a majority of ram will be caching files read often, so it'll still run fast.

With Flashybrid and the 'prototype' PlugPBX builds being ran right now, once the system boots the SD card is mounted read-only, and any tasks that need write/update access to files have their directories bind mounted to a ram disk. Works slick. Flashybrid scripts do all the heavy work, setting up the mounts, syncing changes back to SD card on system shutdown (or can be triggered at any time using the fh-sync script provided. They even have a 'mountro' and 'mountrw' script that will toggle the entire SD card to read or read/write mode.

So as a PlugPBX user, users can boot the system iron clad. BUT, they can ssh in, issue 'mountrw' and now update the system, modify any file, do whatever they want and when done, issue a 'mountro' command.

For typical day to day use, login, make changes to say FreePBX which updates a SQL database and then when done issue a 'fh-sync' command which commits via rsync changes in the RAM disk back to the SD memory card. It works slick as hell.

You even define what directories are to be 'aliased' to the RAM disk, again, the flashybrid scripts do all the hard work. Its very slick.

I've detailed it on the PlugPBX forums, come have a read...

http://forums.plugpbx.org/index.php?topic=32.0

I have yet to find a better solution than this. Its perfect. The stability of say a router type device, but the ability at the flick of a switch to work with it as a regular system, updating, changing and hacking it - and then locking it back down. No more worries about SD card wear leveling issues. I call it lazy mans embedded system design. I'm really really, Lazy.... but also smart....

[bnms]

I tried to use the dependencies with the init script to get flashybrid executed at the right place in the boot process automatically.

Cool -- sounds like a completely feasible solution for root file systems on SD cards!

...

It should be possible to use the Default-Start and Required-Start sections to survive runs of update-rc.d. This should keep you safe until flashybrid is udpated. The next step would probably be to submit a defect with the Debian folks to have this done per default -- given all the problems when starting flashybrid later, doing so sounds like a bug to me...

I use Debian Squeeze on an SD card. I changed the /etc/init.d/flashybrid script to read:

Code:

### BEGIN INIT INFO
# Provides:          flashybrid
# Required-Start:    $local_fs
# Required-Stop:     $local_fs
# Should-Start:      $local_fs
# Should-Stop:       $local_fs
# X-Start-Before:    $network
# X-Stop-After:      $network
# Default-Start:     S
# Default-Stop:      0 6

Thus, the script should be started before networking starts. Then, I removed all the existing links in the other run levels and let insserv recalculate the dependencies:

Code:

cd /etc
rm rc{0,1,2,3,4,5,6}.d/*flashybrid
insserv -v flashybrid

Now, my /etc/rcS.d directory contains:

Code:

...
S14flashybrid
S14procps
S14udev-mtab
S14urandom
S15ifupdown
S16networking
...

Which is where I wanted it to be 

[PlugPBX]

YAY! You made my day  

Code:

### BEGIN INIT INFO
# Provides:          flashybrid
# Required-Start:    $local_fs
# Required-Stop:     $local_fs
# Should-Start:      $local_fs
# Should-Stop:       $local_fs
# X-Start-Before:    $network
# X-Stop-After:      $network
# Default-Start:     S
# Default-Stop:      0 6

Thus, the script should be started before networking starts. Then, I removed all the existing links in the other run levels and let insserv recalculate the dependencies:

Code:

cd /etc
rm rc{0,1,2,3,4,5,6}.d/*flashybrid
insserv -v flashybrid

Now, my /etc/rcS.d directory contains:

Code:

...
S14flashybrid
S14procps
S14udev-mtab
S14urandom
S15ifupdown
S16networking
...

Which is where I wanted it to be 

[mundhra]

pretty awesome, bnms. thank you.

[oliphaunt]

Is everyone concerned still happy with the combination SD + Flashybrid?  Would the problem of mmcqd eating up almost 100% cpu, which I'm currently experiencing after unpacking the debian lenny tarball on SD, also be cured by this solution?