How to clone a SheevaPlug?

[snake]

Hi SonicBoom,

Maybe you could try hacking the script from cbxbiker61 to produce a tarball in .tar.gz format. Rename the created file to rootfs.tar.gz and use it with the sheevaplug-installer.

See this link for additional info:
http://www.plugcomputer.org/plugwiki/index.php/SheevaPlug_Installer


Hope this helps. 

[SonicBoom]

@pingtoo

Ok, I'm not feeling sorry anymore! 

@fragfutter

Thank you for your kind help, but it's like we are speaking different languages!  I'm understanding just a little bit of what you are telling me, and anyway I don't imagine how to replicate it using the sheevaplug! Anyway, thank you again!

@snake

Your solution seems to be the simplest one: I'll try it and post the results as soon as possible! If everything will work fine, you saved my day!

[SonicBoom]

IT WORKS!

Thank you so much to everybody!
Anyway, just to let you know:

1. I used the script provided by pingtoo, just using -z instead of -J and creating a .tar.gz file containing the whole NAND filesystem.
2. I copied the rootfs.tar.gz file created in the point 1. on my USB stick, overwriting the original rootfs.tar.gz provided with the installer-1.0.
3. Following the installer-1.0 procedure, I installed the cloned filesystem coming from one sheevaplug onto another one.
4. Well done.

Many thanx to everybody, especially pingtoo and snake!

[feffer]

What works well for me is to configure the sheeva with the root filesystem on SD.  Then when I want to backup the sheeva I shutdown, put the SD in in my notebook computer and tar up the SD image.  Restoring the SD is just a matter of formatting an SD and untarring my backup to the SD.  This makes it extremely easy to clone the sheeva.

Here's the script I run (on the notebook) to backup my SD.  The SD is mounted at /mnt/tmp.

#! /bin/bash
date=$(LC_ALL=C date +%Y.%m.%d)
sudo tar c -C /mnt/tmp --exclude=lost+found -J -v -f /var/tmp/SheevaRoot-$date.tar.xz .

I also want to backup my SD. I have debian on it using Martin Michlmayr's howto. This creates separate /boot and / rootfs. Using a card reader and dd, I created an image by

Code:

dd if=/dev/sdc of=~/sdcard.img

and then restored it to an identical SD card with

Code:

dd if=~/sdcard.img of=/dev/sdc

The cloned SD got the same UUID's and booted fine -- dd got the whole card with /dev/sdc even though there were 3 partitions (sdc1, 2 and 3). However, it took a long time -- 20 min to create the image and 50 min to restore it to the new card. I've used tar only a little -- it would probably be faster and I could then use it to backup minor changes to debian. My only concern is getting the formatting right and the partition table etc. How would I do that? Or is there a way to get dd to work faster?

EDIT: The process detailed above works fine, but there is a better, faster way to maintain a backup SD card than using dd each time. After making and testing the initial cloned SD card, use rsync to periodically copy the contents of the / and /boot partitions to a hdd. A backup SD card cloned as above can be synced fairly quickly from the hdd backup. Be sure to use rsync with the "-a" option to preserve ownership and permissions. This seems to work pretty well, but if there's a better way to do this, please tell.

Regards,
feffer

[kveroneau]

An efficient way of cloning SheevaPlugs is over a network using SSH and Pipes.

For this to work, you will need to have a network configured initrd booting from either a storage device or TFTP.

This solution would work for currently configured plugs in Sheeva_Installer, but you need to quickly update a bunch of plugs without using the USB key in each plug all over again...

Make a special host initrd, and then guest initrds to make it even simpler.  Boot your development plug into the host initrd image and mount the UBIFS in say /mnt, and make sure the SSH service is running.

Boot a production plug into the guest initrd which will automatically SSH to the host plug with a static IP.

SSH example command: ssh -n root@hostplug 'cd /mnt; tar zcvf - *' | tar zxvf -

Make the guest plug do a UBIATTACK, MOUNT, and CD into the destination directory before running the above SSH.  You may also want to run the UBIFORMAT commands as well.

Feel free to remove the 'z' parameter in TAR if your network is very very fast, as this will speed up the process.

I would never recommend transferring a LIVE file system, one which has been mounted via bind, as this may cause some applications and services to act funny.  I tend to only TAR Linux file systems which have been safely halted to have the best possible state when cloning using tarballs.  But that's just my personal phobia.

Overall, if an initrd is scripted right, you could adapt the SheevaInstaller to obtain the rootfs.tar.gz from an SSH source such as this for installing fresh off the shelf plugs.  Be sure to place your current uImage on the USB stick as well.

However, you can easily burn the TFTP'd uImage to the NAND before loading your guest initrd to clone the plug.  There are also u-boot commands for running special scripts from a TFTP which could automate the u-boot upgrade process and avoid using OpenOCD completely.  If you make the u-boot NAND partition visible in the host and guest initrd systems, you could easily transfer the entire u-boot binary and environment over the network and burn it directly to the NAND of the new plug!

For an even bigger kicker, when booting your custom guest initrd, make the MTD partition one huge single partition which can be accessed under Linux.  Use the SSH piping above to transfer an exact 512MB image over the network and burn it on a new plug!!!  Now that's mass-production at it's best.  For this, it might even be best to have the 512MB image stored on a speedy server machine to serve the images to all the hungry guest plugs to feed from.

In effect, if you choose the last option, for each new plug you purchase all you need to do is:

1.  Plug in your pre-configured USB stick with kernel and initrd into the new plug.
2.  Boot the plug using the USB stick's kernel and initrd.
3.  Wait while the initrd system transfers the entire 512MB NAND image from the server/devplug and burns it.
4.  Reboot the plug and enjoy the exactly cloned plug(u-boot, environment, mtdparts, etc...)

Hope this helps anyone who need to mass-produce plugs for the market.

[bkuker]

@Eagle,

You are correct, it is not easy to use dd to access NAND. However your origin post does not make it clear hence my question. I was trying to understand what is your plug condition.

Can anyone explain why this is difficult? It looks like /dev/mtdblock1 is the kernel image, and /dev/mtdblock2 is the root filesystem. I would naively assume I could dd both to images on my bootable USB stick, boot a fresh plug from same USB, then dd those images back onto the new plug's flash?

[bkuker]

You are correct, it is not easy to use dd to access NAND.

Can anyone explain why this is difficult?

To answer my own question it seems that MTD (Memory Technology Devices) are not quite normal block devices, and jffs is aware of that. Jffs will not run on top of anything but an MTD, and you can not simply DD a jfs image on and off. It does work under some circumstances I do not understand, but not in the common case.

You can DD off of an MDT (from /dev/mtdblockN), and then mount that image (via a loopback device AND a weird block to MTD converter) but you can't, as far as I can tell, DD onto an MTD. I expect that a UBIFS filesystem will work the same way.

[restamp]

I don't claim to have any special knowledge of the MTD subsystem, but I wonder if part of the problem with successfully writing the NAND memory involves choosing the proper block size for the device.  Here is what I get if I "cat /proc/mtd":

$ cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00100000 00020000 "u-boot"
mtd1: 00400000 00020000 "uImage"
mtd2: 02000000 00020000 "rootfs"
mtd3: 0db00000 00020000 "data"
$

As you can see, the erase size (block size) of the NAND is rather large -- 0x20000 or 131072 bytes, or 256 512-byte blocks.  'dd' defaults to a 512-byte block.

I don't particularly want to mess with my NAND, but those of you who are might try using "dd bs=128K ...".  If anyone does this, I'd be interested in hearing if it work, or changes things.

The other inherent problem with 'dd' is that its behavior upon encountering an error does not work well with NAND.  My understanding is that NAND errors are to be expected, and when they are encountered, that block is skipped and the contents written to the next good block.  But, 'dd' doesn't do this, and as far as I know, it has no way to deal with NAND write errors appropriately.  Thus, it is a poor tool for this task.

Again, I have no special knowledge of how the MTD subsystem is implemented in practice, so this may not be the problem at all.  Perhaps the subsystem handles 512-byte writes.  Perhaps it silently handles NAND block errors.  It would be nice is someone who knew would answer us here.  This is where I'd start looking, though, if I were trying to investigate this problem.