Is Flash memory NOT used in running system - Always run out of DDR2 ?

[superpat]

Hi,

I am getting very old, it has taken me a long time to realise that the Marvel Plug has a very different mode of operation to ARM systems with Flash memory I am familiar with.

The plug has 512Mb of Flash memory and 512Mb of DDR2 memory.

I made the assumption that the DDR2 memory was for user space and the system ran out of flash.  WRONG!   

Reading through the Marvell documents  it becomes clear that the  initial task of the cpu is to copy the image from the "place where it is stored" and MOVE it to DDR2 memory and run it in the DDR2 memory.

This DDR2 memory is the same as in your own pc, It is volatile but non destructive and there are no problems with wearout.

By "place where it is stored"  I mean Flash, Serial uart, PCI express, SPI  (SDcard), Sata.   

Booting from SDI card can either run the image from the SDcard on move it to the DDR2 memory.

This makes  for a very interesting question :---

Assuming the "image" moved to DDR2 is the complete Linux image, file system and kernel,  (It only talks generically of "image" in the Marvell data sheets), then

WHAT TYPE OF FILE SYSTEM SHOULD BE NORMALLY EMPLOYED? 

If the image booted from the boot medium is moved to and is running out of DDR2, then ANY of the Linux file systems  could usefully be used. NOT just Jffs2.

The only time a JFFS2 file system need be used is for the case where the image is NOT moved out of the SDcard but is run from the sd card itself.

I may be  wrong, but it is a very interesting concept. 

The Marvell document is  FS_88F6180_9x_6281_OpenSource.pdf  ( from the Marvell website. ) There is a long section on booting  (24.2)

from PARAGRAPH 24.2.4

Boot Image Format

The boot image is the binary image residing on the specific boot device. It must contain a valid main
image header at offset 0x0 and may include a header extension, according to the header extension
flag in the main header. In addition, it includes the binary image that should be copied and executed
in the SDRAM (excluding boot from SPI when the image is executed directly from the SPI memory
space).
In most cases, the header extension is used to provide the bootROM firmware with the DDR
configurations. It must be concatenated to the main header, and the total size of the main header
and the header extension must be exactly 512 bytes. The header extension may be omitted if the
boot is to be performed directly from the boot device, without copying the user image to the DDR
(valid for SPI boot device only).
The source image can immediately follow the main header or header extension, or it can reside at a
more distant offset (this option is necessary in boot from SATA) (see Figure 80).

AM I correct in the above ?

Is this written down in any specific Plug documentation?

regards

Patrick

[cbxbiker61]

You're question is a little hard to follow, but I think I get the jist of it.  Here is what I think you're looking for.

There are three sections in flash.  1. U-Boot, 2. Kernel image, 3. JFFS filesystem.

On bootup:
1. U-Boot copies itself from flash into ram and continues execution from ram
2. U-Boot copies the kernel image from whereever (typically flash) to ram
3. U-Boot executes the kernel from ram (the kernel reclaims the ram previously used by U-Boot)
4. The kernel reads the JFFS filesystem from flash (this part is NOT copied to ram, but is accessed directly from flash)

In a nutshell, you can't use any old filesystem for the flash filesystem.  JFFS is specifically designed to R/W files stored in flash.

[superpat]

@cbxbiker61

Thanks for your reply.

Please have you got any pointers to Marvell literature regarding the uboot and kernel only copy sequence

I still need convincing that we cannot run the root and user fs in DDR2 .

There is 512Mbytes of Nand flash on the plug, This has to contain the complete Uboot binary,  uImage (kernel) and the root and user file systems. (assuming no external storage)

There is also 512Mbytes of Ram in the plug.   Therefore there is no reason why the complete "Image" i.e the kernel and the root and user file systems cannot be copied over to the RAM and the  complete system run from ram, or am I missing something here?   

According to the Marvell information the limit of transfer size is (quote) "The image can be copied up to offset 0x17FFFFFF (384 MB) in the DDR".

What I am saying is that seems that to just run the kernel in ram, means there must be a lot of free space left! 

The kernel uImage for the plug is only  around 2.0 Mbytes!!    I cannot believe that the other 510 mbyte of ram is left unused.

regards

Patrick




 

[plugit]

Oh, it's not unused. Linux caches as much as it can. Further - you'll notice that there's no swap at all configured out of the box with the SheevaPlug, so you'd better have some spare RAM available to prevent problems!

There are a few things in RAM, though:

root@rlyeh:~# mount | grep tmpfs
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
varrun on /var/run type tmpfs (rw,nosuid,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /var/cache/apt type tmpfs (rw,noatime)

All the tmpfs stuff is RAM-based.

I'm pretty happy with my current scenario: uImage on NAND, rootfs on SDHC, and storage on USB.

root@rlyeh:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
rootfs                3.8G  526M  3.1G  15% /
tmpfs                 252M     0  252M   0% /lib/init/rw
varrun                252M  476K  252M   1% /var/run
varlock               252M     0  252M   0% /var/lock
udev                  252M  112K  252M   1% /dev
tmpfs                 252M     0  252M   0% /dev/shm
tmpfs                 252M   29M  224M  12% /var/cache/apt
/dev/sda3             504G   32G  448G   7% /storage
/dev/sda4             392G  199M  372G   1% /mnt

[karurosu]

Just as they mentioned, you NEED the kernel in RAM (and modules maybe too), however there is no need to have everything there, especially since (as plugit mentioned) there is no swap configured.
All the free unused RAM is there so you can use it to run other software. Personally I would try to keep very few kernel related things in RAM and use it for other software (HTTP servers, MySQL, etc).

Now, if you want to run everything from RAM, live distros have been doing that for years, so you can look for a way to mount your root into a ramfs, but as I said in the above paragraph I dont see much point in doing it.

[dg]

What you're looking for is the ability to map the flash into the CPU's address space, which allows you to run code directly out of flash. While you can do this with NOR flash, you can't do this with NAND flash --- it just doesn't work like that. All you can use NAND flash for is as a filesystem, which means you've got to copy the data out of it and into RAM before you can run it.

This is typically what you want anyway, because flash is very slow compared to DRAM and will slow execution down considerably.

I don't know for sure how the SheevaPlug works, but other systems like it I've seen typically have a small amount of NOR flash that's used for booting --- NOR flash can be mapped and works just like ROM. This then contains the code necessary to read uBoot out of the NAND flash and run it.

[Rabeeh Khoury]

You're question is a little hard to follow, but I think I get the jist of it.  Here is what I think you're looking for.

There are three sections in flash.  1. U-Boot, 2. Kernel image, 3. JFFS filesystem.

On bootup:
1. U-Boot copies itself from flash into ram and continues execution from ram
2. U-Boot copies the kernel image from whereever (typically flash) to ram
3. U-Boot executes the kernel from ram (the kernel reclaims the ram previously used by U-Boot)
4. The kernel reads the JFFS filesystem from flash (this part is NOT copied to ram, but is accessed directly from flash)

In a nutshell, you can't use any old filesystem for the flash filesystem.  JFFS is specifically designed to R/W files stored in flash.

There is a missing item '0' which that the 88F6281 (aka Kirkwood) chip used in the plug has internal bootrom, that reads a reset strap from the board and understands it needs to boot from NAND (where U-Boot resides).
So the 512 byte header mentioned are the first 512 byte header of U-Boot that contains instructions how to basically setup DDR and other stuff to start booting U-Boot.
Afterwords steps '1..4' as mentioned above applies.