• Home
  • Help
  • Search
  • Login
  • Register
  Show Posts
Pages: [1] 2
1  Hardware and U-Boot firmware / U-Boot stuff / Re: uboot-envtools on: July 21, 2010, 09:06:01 AM
When I run
root@debian:/# sheeva-ubootenv-print > bla
I get the following error:
sheeva-nandrs-decode: too many errors in block #0 of chunk #0
sheeva-ubootenv-decode: environment must be 128Kb.

Is that a Sheeva or a Guru? I'm having the same trouble as you with my Gurus, but it works fine on my sheevas! It seems something has changed in the way the environment is being saved: altough u-boot itself is stored using 4-bit ECC, it stores the environment using 1-bit ECC, in the same partition, which seems really ugly. This means that you can read the memory that holds the environment from Linux just using dd (which you can't do when the data is stored with 4-bit ECC), but so far I have been unable to write the data back, it becomes garbled both with dd and with nandwrite -q -n -s 0xa0000 /dev/mtd0.
2  Hardware and U-Boot firmware / U-Boot stuff / Re: uboot-envtools on: July 20, 2010, 06:18:46 PM
Sorry for the dumb question: Where can I download this tool. I can't find a download link on the google code site!

Sorry for the delay, I haven't been around here in a while. I haven't made a package yet... you must download the source and compile it yourself.

3  Hardware and U-Boot firmware / U-Boot stuff / Re: uboot-envtools on: March 18, 2010, 03:14:31 PM
Actually nandwrite does not do erase.

If it didin't, it wouldn't be writing. You can't write to NAND if you don't erase first. Either nandwrite is erasing the NAND before writing to it, or the kernel is doing so for it. But you can be sure that someone is erasing the NAND just before writing data to it.

nandwrite does work but I want to make system that will fit in my initramfs, if I to include some part of mtd-utils and my own code it become too much to work.

nandwrite itself is not too large: about 12KB. Doing a smaller program for the same purpose is going to be tricky.

P.S. I would leave the u-boot upgrade to JTAG itself  Cheesy

Wise man! But well, for those days when one feels adveturous and in the mood to brick a plug for no reason at all... :-)
4  Hardware and U-Boot firmware / U-Boot stuff / Re: uboot-envtools on: March 18, 2010, 12:21:07 PM
I think in your step you miss the process to erase NAND.

Not really: nandwrite takes care of that. The beauty of this solution is precisely that it avoids the complexities of NAND handling, something at which mtd-utils is very good.

I want to further develop this to support kernel/initrd upgrade

The sheeva-nandrs-encode program in my package should make it possible to upgrade a plug's u-boot from a running kernel. As to the kernel image/ramdisk... can't you just do that with nandwrite?

I would expect nandwrite -p /dev/mtd1 kernel-image-file to just work, assuming that kernel-image-file is in the right format and is not large enough to overwrite the start of the ramdisk image.
5  Hardware and U-Boot firmware / U-Boot stuff / Re: uboot-envtools on: March 18, 2010, 05:52:51 AM
At the current time, to my knowledge, no one has been successful at updating the Uboot environment from a running kernel.

This state of affairs changed yesterday :-)

You can use these tools I wrote for this purpose https://code.google.com/p/sheeva-uboot-tools/

And thanks to the participants of this thread, you guys gave me all the right clues on how to solve it!
6  Hardware and U-Boot firmware / U-Boot stuff / Re: uboot 3.4.19 - need to change/add uboot parameters without serial console on: March 18, 2010, 05:48:47 AM
to change your u-boot env vars you have to build the env tools in the tools/env folder. I've done this for a coldfire platform a year ago and it works fine. When the build was successful there should be a binary fw_printenv if you set a link to fw_setenv you could also write the u-boot variables. Take a look into tools/env/fw_env_main.c.

Unfortunately, this won't work on sheevas (not to mention that fw_{print,set}env are really ugly).

As of yesterday, however, you can do this with these tools I just hacked together: https://code.google.com/p/sheeva-uboot-tools/.
7  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 17, 2010, 10:14:33 PM
Have a blast: https://code.google.com/p/sheeva-uboot-tools/.
8  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 17, 2010, 05:56:34 AM
Status report: I am now able to go from raw data + OOB data to a plain text representation and back!

I'm cleaning up and packaging the code to release it ASAP.
9  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 16, 2010, 05:47:47 AM
Good news!

The file nand_ecc_rs.c indeed contains the algorithm used to compute the 4-bit ECC in /dev/mtd0. Using that, I have written a program that takes the output of nanddump -n -s 0xa0000 -l 0x20000 /dev/mtd0 and successfully performs error correction, checksum validation and conversion to a newline-separated text file.

In the process of getting this done, however, I noticed that Marvell has been less than careful about licensing: the header in nand_ecc_rs.c first claims that the file "contains an ECC algorithm from Toshiba that detects and corrects 1 bit errors in a 256 byte block of data," then goes on to say that the file is released under GPLv2 or later. However, the first claim is patently wrong: this is certainly not 1-bit ECC, it appears to me that the comment is there simply because it was in nand_ecc.c and someone forgot to change it. The same applies to the comments at the top of the encode/decode functions: they are identical to their counterparts in nand_ecc.c, and completely fail to properly describe the functions they supposedly document.

It seems to me to have been (very lightly) adapted from code available at The ECC Page (http://www.eccpage.com/), as an illustration of a Reed-Solomon codec (http://www.eccpage.com/rs.c). The program is distributed with the following notice:

   This program may be freely modified and/or given to whoever wants it.
   A condition of such distribution is that the author's contribution be
   acknowledged by his name being left in the comments heading the program,
   however no responsibility is accepted for any financial or other loss which
   may result from some unforseen errors or malfunctioning of the program
   during use.
                                 Simon Rockliff, 26th June 1991

So, as far as I can tell, Marvell is distributing this code in violation of the author's terms :-(

So, I'm off to write the textfile-to-checsummed-environment code! Stay tuned...
10  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 15, 2010, 10:44:16 AM
I think this may be a misunderstanding or wrong terminology. the fw{print,set}env using read/write system call to do its I/O and I see no evidences about skipping OOB blocks so that is why I think it is the problem.

That is why it's called "out-of-band" data: read/write system calls by themselves will skip the OOB data. If you want to read/write OOB data, you must do it thorugh the MEMREADOOB respectively MEMWRITEOOB ioctl.

Since /dev/mtd0 is block device (I think) may be we can still do my suggestion with /dev/mtdchar0?

No, you are thinking of /dev/mtdblock0. /dev/mtd0 is a character device.

do you have u-boot source from CD or download from wiki? I see there is a drivers/mtd/nand/nand_ecc_rs.c may be that is where the 4 bit calculation code is. I read the file but it is beyond my little head can understand Grin

I got it from denx's git repository.  I just downloaded the sources from Marvell, and indeed, there it is! It's an implementation of Reed-Solomon, a popular ECC, which fits the interpretation of the data. I'll be working on the filters, I'm confident that I'll have a solution available soon!
11  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 15, 2010, 09:53:25 AM
Anyway, I think I figure out the problem why the fw_setnv/fw_printenv is not working for MTD from Linux. it is because the utility is read/write the mtd device as raw stream without consider the OOB blocks. so the end result will always have CRC error because extra OOB data in the mix.

No, that is definitely not the problem. fw_{print,set}env are not reading /dev/mtd0 in raw mode, and they don't get to see the OOB data. This can be seen both in the data that they read, and in the fact that whenever you use the to access the NAND, the operating system complains about unrecoverable errors once for each 2KB-block (you can see the messages in the console, or in dmesg).

This is an indication of the real problem: those programs attempt to read the data in normal mode, so the kernel tries to use the ECCs it expects in the OOB data to correct eventual errors. But the kernel expects a different ECC than the one that is actually stored in the OOB data, and so it detects an unrecoverable error --- but not before it has attempted a correction or two, so the data that the programs get is actually different from the data that is stored, hence the CRC mismatch.

All of this isn't guessing: I have actually compared the data that fw_printenv gets with the data read by nanddump -s 0xa0000 -n -o /dev/mtd0, and they are different.

I propose we simply skip finding ECC calculation routines instead we let existing utility do the work. we can use program from mtd-utilis to dump the content from mtd0 then use the sheeva-installer supplied fw_setenv to modify the dump then use nandwrite to put modified dump back to mtd0.

This is the course of action I am pursuing at the time, but I am afraid that it won't be so simple. Here's what works, and what doesn't.

  • To read the environment data, we can simply do nanddump -s 0xa0000 -n -o -l 0x20000 to dump it to stdout. This is the Really Simple Approach, because it doesn't take into account the possibility that the NAND may indeed have flipped a bit, in which case we are in deep doo-doo. The better solution would be to use nanddump -s 0xa0000 -n -l 0x20000 to get the data with OOB data inline, and feed that to a filter to interpret the ECC and correct any errors it finds, and maybe even check that the CRC is kosher.
  • Either use dd, tr and sed to convert the output of the last command into a text file that can be edited with your favourite editor, or let the filter described in the previous point do this job.
  • Once the file looks the way you want it, re-format it into a 128KB file, with leading CRC32 in big-endian order at the first 4 bytes, the environment immediately afterwards, and padded with 0x00, and write that using nandwrite. But this won't work if we attempt the simplistic approach of just doing nandwrite -x 0xa0000.

The problem with the last point is that if we attempt to write the data like that, nandwrite will write the data with a 1-bit ECC, while u-boot expects 4-bit ECC... we have the opposite problem than we had at the beginning: we can read the data, but U-Boot can't. We could use nandwrite -n to write the data without ECC, but that will leave the NAND in an inconsistent state, which u-boot will not be able to read either.

The only clean way I see out of this is to compute the ECC ourselves in user space, and use nandwrite -o to write both the payload and the precomputed OOB data to the NAND. The program that does this could also do the reformatting, padding and CRC computation described in the last step.

This makes everybody happy, the only thing standing between me and having working code is the fact that I still don't know how the ECC is computed (and no: the functions in drivers/mtd/nand/nand_ecc.c are no help, they seem to be doing 1-bit ECC).

If we figure that out, we're there.
12  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 15, 2010, 07:26:30 AM
Again, trying to figure out the layout by squinting at an hex dump of the OOB data that is present every 2048 bytes, it looks to me like it contains:

  • 24 bytes set to 0xff, regardless of the payload values;
  • 40 bytes of ECC data, apparently structured in 4 10-byte blocks, each corresponding to a 512-byte data block. The ECC for 512 contiguous 0x00 bytes seems to be all zeros, the ECC for all 0xff seems to be all 0xff.

This is consistent with info I gathered by searching the web for "nand OOB layout 4-bit ECC" (and about a dozen other combinations of search terms), although it would seem to me that those 24 initial bytes set to 0xff ought to be put to some use, but maybe they are only used if the sector is bad or something.

By the way, only data in /dev/mtd0 seems to have this layout: /dev/mtd1's and /dev/mtd2's OOB data seems to have an entirely different layout, which is consistent with the theory that /dev/mtd0 uses a different ECC than the rest.

Now, all we need is to identify the ECC, and algorithm to implement it. Of course, the code to do this ought to be available inside u-boot, since it can read/write the environment, but I haven't been able to find where in u-boot this computation is done :-(

13  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 14, 2010, 05:13:47 PM
I have confirmed the suspicion that the CRC mismatch is due to ECC, so we should be a very small distance away from a solution. However, I can't seem to find any reference about which ECC is used, nor how it is encoded in the NAND's 64-byte OOB sector. There's a file called drivers/mtd/nand/nand_ecc.c within u-boot sources, but from what little I could gather from the comments, it seems to be a 1-bit ECC, not the 4-bit ECC we are looking for.

If anyone knows about how to compute and encode the OOB data, I'm sure we could have a working solution within hours.

14  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 14, 2010, 12:28:21 PM
I think it is crc7, I check u-boot code.

I seriously doubt it: I couldn't find even a reference to crc7 in the u-boot source code.

Yet, we are getting closer!

The reason why fw_printenv computes a different CRC than u-boot is because they are operating on different data, because of read errors due to using the wrong ECC mechanism. From what I can read in another thread http://plugcomputer.org/plugforum/index.php?topic=117.msg654#msg654, the problem is that u-boot handles the NAND with 4-bit ECC, while Linux tries to handle it with 1-bit ECC, because the latter seems to be more reasonable for the specific type of NAND used in the Sheevaplug (don't ask me why: I'm basically rephrasing te contents of that post, I am only beginning to understand how this whole NAND business works). This ECC information seems to be stored inside some out-of-band (OOB) data that the NAND memory sets aside for each block.

Fortunately, there is a solution: the package mtd-utils has handy utilities such as nanddump and nandwrite that allow us to do I/O to the NAND in raw mode, bypassing the kernel's ECC logic and enabling access to the OOB data, which is normally hidden from the user. With this utilities, we can dump of the environment's contents, edit it to our hearts content, then write it back.

The only problem: when we write it back, we need to do so with 4-bit ECC, so u-boot will be able to read it. I haven't found a way to tell nandwrite that it should write with 4-bit ECC (it only has a flag to turn ECC on or off), but maybe it's there, and I'm too dumb to find it. Even if it's not there, we can still compute the whole ECC thing by ourselves in the user side, and write the data back with pre-computed ECC and all in raw mode, which nandwrite supports.

The only problem now is that I still haven't found any reference on how this 4-bit ECC is supposed to be computed and stored, but as soon as we get that sorted out, we should be good to go.
15  Hardware and U-Boot firmware / U-Boot stuff / Re: fw_printenv offsets on: March 13, 2010, 06:12:14 PM
I'm still trying to figure out why u-boot and fw_{print,set}env compute different CRCs for the same environment.

In the meantime, I found out that fw_printenv is computing crc32 correctly. So it seems that u-boot does not use crc32 but another checksum algorithm?
Pages: [1] 2