Forum Topic - ETFS: sporadic total data loss when writing big files?: (3 Items)
   
ETFS: sporadic total data loss when writing big files?  
Hi,

we're using ETFS on an mpc8313 with a 2K NAND. The BSP only supports 512Byte-NAND, which is why we extended it to 2K 
NAND by integrating code from the mpc5121e BSP.

When we write large files (> 3 MB) to ETFS and at a later stage delete them again, sometimes this will lead to a total 
data loss: after rebooting upon mounting the FS the ResMgr reports data errors (CRC32 not matching) or a corrupted/
invalid filesystem. Usually this means that all files contained in the FS are lost.

Any ideas on what may be the cause or where to start looking? Have others encountered this problem too (i.e. libetfs or 
general BSP bug) or is this a problem unique to our port of the 2K NAND access?

Thanks,
 Marc
Re: ETFS: sporadic total data loss when writing big files?  
I've dug a bit deeper...

We use 2K Nand (1 GB), with ETFS Partition 2 starting at page 473088 (i.e. etfs called with "-r 946176"). With 2K Nand, 
each block is 64 pages. On ETFS, each page is 2K and equals one ETFS cluster.

We modified the ETFS Transaction Read function of the BSP to print debugging information (you will see those included 
further below as "reference A" and "reference B"). Then the following steps were executed on the target (8313):
1. format NAND ("nand erase" from UBoot to have a clean system)
2. etfs-format NAND (etfsctl -d /dev/etfs2 -f)
3. reboot, ensure the FS is up and running
4. write a file of 10.981.563 Bytes (as reported by windows/cygwin) to /tmp (via Momentics/qconn), then do a "cp" to /fs
/etfs
5. after writing the file has succeeded, reboot. The readtrans debug printfs are reference 'A'. Make sure that the file 
is properly contained in ETFS (by reading it back and verifying)
6. create four or five empty files in /fs/effs by using "touch"
7. reboot.  The readtrans debug printfs are reference 'B'.

During reboot, etfs prints "Truncating fid 0x6 from 10907648 to 0 bytes". After reboot, all files (large file and the 
small touched files) are gone, so the filesystem is empty! From the readtrans debugging printfs, it seems fid 0x6 is the
 large file. The small touched files are not even reported as lost during reboot, they just seem to vanish.

Here are some details:
(i've sometimes added a comment "new block" for seeing better where the block boundaries are)


"Reference A" (Booting after the large file has been written... edited for brevity)

(Start of ETFS)
cl 473088	ETFS_TRANS_ERASED
cl 473152	ETFS_TRANS_ERASED
cl 473216	ETFS_TRANS_ERASED
cl 473280	ETFS_TRANS_ERASED
cl 473344	ETFS_TRANS_ERASED
...
cl 473408	OK (seq 0x1 fid 0x1 clusters 0x0 nclusters 0x1
cl 473409	OK (seq 0x1 fid 0x1 clusters 0x0 nclusters 0x1
cl 473410	OK (seq 0x1 fid 0x6 clusters 0x0 nclusters 0x10
cl 473426	OK (seq 0x1 fid 0x6 clusters 0x10 nclusters 0x10
cl 473442	OK (seq 0x1 fid 0x6 clusters 0x20 nclusters 0x10
cl 473458	OK (seq 0x1 fid 0x6 clusters 0x30 nclusters 0xe
(new block)
cl 473472	OK (seq 0x2 fid 0x6 clusters 0x3e nclusters 0x10
cl 473488	OK (seq 0x2 fid 0x6 clusters 0x4e nclusters 0x10
cl 473504	OK (seq 0x2 fid 0x6 clusters 0x5e nclusters 0x10
cl 473520	OK (seq 0x2 fid 0x6 clusters 0x6e nclusters 0x10
....
cl 476160	OK (seq 0x2c fid 0x6 clusters 0xabe nclusters 0x10
cl 476176	OK (seq 0x2c fid 0x6 clusters 0xace nclusters 0x10
cl 476192	OK (seq 0x2c fid 0x6 clusters 0xade nclusters 0x10
cl 476208	OK (seq 0x2c fid 0x6 clusters 0xaee nclusters 0x10
(new block)
cl 476224	OK (seq 0x2d fid 0x6 clusters 0xafe nclusters 0x10
cl 476240	ETFS_TRANS_ERASED
(new block)
cl 476288	ETFS_TRANS_ERASED
(new block)
cl 476352	ETFS_TRANS_ERASED
(new block)
cl 476416	OK (seq 0x30 fid 0x6 clusters 0xbbe nclusters 0x10
cl 476432	ETFS_TRANS_ERASED
(new block)
cl 476480	ETFS_TRANS_ERASED
(new block)
cl 476544	ETFS_TRANS_ERASED
(new block)
cl 476608	OK (seq 0x33 fid 0x6 clusters 0xc7e nclusters 0x10
cl 476624	ETFS_TRANS_ERASED
(new block)
cl 476672	ETFS_TRANS_ERASED
(new block)
cl 476736	ETFS_TRANS_ERASED
(new block)
cl 476800	ETFS_TRANS_ERASED
(new block)
cl 476864	ETFS_TRANS_ERASED
(new block)
cl 476928	ETFS_TRANS_ERASED
(new block)
cl 476992	ETFS_TRANS_ERASED
(new block)
cl 477056	OK (seq 0x3a fid 0x6 clusters 0xe3e nclusters 0x10
cl 477072	OK (seq 0x3a fid 0x6 clusters 0xe4e nclusters 0x10
cl 477088	OK (seq 0x3a fid 0x6 clusters 0xe5e nclusters 0x10
cl 477104	OK (seq 0x3a fid 0x6 clusters 0xe6e nclusters 0x10
....
cl 478656	OK (seq 0x53 fid 0x6 clusters 0x147e nclusters 0x10
cl 478672	OK (seq 0x53 fid 0x6 clusters 0x148e nclusters 0x10
cl 478688	OK (seq 0x53 fid 0x6 clusters 0x149e nclusters 0x10
cl 478704	OK (seq 0x53 fid 0x6 clusters 0x14ae nclusters 0x10
(new block)
cl 478720	OK (seq 0x54 fid 0x6 clusters 0x14be nclusters 0x10
cl 478736	OK (seq 0x54 fid...
View Full Message
Re: ETFS: sporadic total data loss when writing big files?  
a small addtion..
the dumps of the NAND data have been done after the FS has been started up during the "reference B" printfs. This may 
mean that ETFS has modified the data during the startup (?) as part of its repair process (?). This may explain why the 
dumped data is in contradiction to the info reported by ETFS during startup "B".

But of course this doesn't explain nor solve the bug itself...