Project Home
Project Home		 Documents
Documents		 Wiki
Wiki		 Discussion Forums
Discussions		 Project Information
Project Info
Filesystems/Discussion/General/Partial files being create?/List of Posts
Forum Topic - Partial files being create?: (12 Items)
   
Update
Robert Murrell
08/12/2011 10:13 AM
post88066
Partial files being create?   
I found an unexpected and disturbing issue relating to the QNX 4 file system.  When writing to a file, and then powering
 down during the write, on powerup a partial file is created!  I was expecting the file name to be present, but with a 
zero length.  I would expect that NONE of the data written to the file would be committed until the file was closed.  
Unfortunately, my program was writing to /.boot at the time of the powerdown.  On repowering, the IPL finds the image 
signature in the first 8Kbytes, thinks its good, loads and executes it.

In our application, the system must be tolerant of unexpected powerdowns.  I'm developing under QNX 6.5.0 and using the 
low-level C open-write-close functions.  Is there a way to prevent partial files under this condition?
David Sarrazin
08/12/2011 11:01 AM
post88075
RE: Partial files being create?   
The QNX4 file system is documented as not being power-safe.  You can get
corrupted files, or partial files if power is removed during writing.
This is exactly what the QNX6 power-safe file system was created to
address.

Even in the QNX6 file system, if you are performing a series of write()
calls to write the data, you can still get a partial file if you remove
power after some write() calls have completed.  However, on the QNX6
file system, there is a solution: 

chattr +hold /
<copy your /.boot file>
chattr -hold /

The "chattr +hold/chattr -hold" commands lock-out the QNX6 file system's
"snapshot" operation.  On QNX6, a snapshot changes the current file
system working view into a committed view on disk.  By bracketing your
file operation with that, you can ensure atomicity.  This is documented
here:

http://www.qnx.com/developers/docs/6.5.0/topic/com.qnx.doc.neutrino_sys_
arch/fsys.html?path=13_9_9_6_2#QNX6_performance


David

> -----Original Message-----
> From: Robert Murrell [mailto:community-noreply@qnx.com]
> Sent: August-12-11 10:13 AM
> To: general-filesystems
> Subject: Partial files being create?
> 
> I found an unexpected and disturbing issue relating to the QNX 4 file
> system.  When writing to a file, and then powering down during the
> write, on powerup a partial file is created!  I was expecting the file
> name to be present, but with a zero length.  I would expect that NONE
> of the data written to the file would be committed until the file was
> closed.  Unfortunately, my program was writing to /.boot at the time
of
> the powerdown.  On repowering, the IPL finds the image signature in
the
> first 8Kbytes, thinks its good, loads and executes it.
> 
> In our application, the system must be tolerant of unexpected
> powerdowns.  I'm developing under QNX 6.5.0 and using the low-level C
> open-write-close functions.  Is there a way to prevent partial files
> under this condition?
> 
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post88066
Robert Murrell
08/12/2011 11:27 AM
post88076
Re: RE: Partial files being create?   
Great.  This isn't a problem under FAT file systems, other than lost clusters which are easily recovered.

Now for the next questions:

Does the QNX 6 filesystem need a new IPL?  We are using a custom ipl-diskpc2.

It looks like I use the devctl function to programmatically do what chattr is doing.  Is there any guidance on how to 
set/clear this hold flag using devctl?

The section on "Required properties of the device" is unclear.  Will the QNX 6 filesystem work with a Compact Flash card
?
Mate Szarvas
08/12/2011 11:34 AM
post88077
Re: Partial files being create?   
> Is there any guidance on how to set/clear this hold flag using devctl?

There is a sample here:
http://www.qnx.com/developers/docs/6.4.1/neutrino/user_guide/fsystems.html#Q
NX6_snapshots


On 11-08-12 11:27 AM, "Robert Murrell" <community-noreply@qnx.com> wrote:

> Great.  This isn't a problem under FAT file systems, other than lost clusters
> which are easily recovered.
> 
> Now for the next questions:
> 
> Does the QNX 6 filesystem need a new IPL?  We are using a custom ipl-diskpc2.
> 
> It looks like I use the devctl function to programmatically do what chattr is
> doing.  Is there any guidance on how to set/clear this hold flag using devctl?
> 
> The section on "Required properties of the device" is unclear.  Will the QNX 6
> filesystem work with a Compact Flash card?
> 
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post88076
>
David Sarrazin
08/12/2011 1:09 PM
post88080
RE: RE: Partial files being create?   
Lost clusters under FAT are the same problem, the file is not
complete/incorrect.  It's not possible to recover those clusters if the
boot image is damaged.

A colleague brought up a good point.  The QNX6 file system already
ensures that /.boot and /.altboot will never be partially written, by
implicitly doing the "+hold".  All other files require the chattr
command.

Yes, there's a new second-stage IPL required (diskpc2-fsq6) for the QNX6
file system.

The "required properties" question.  The QNX6 file system is, above all
else, supposed to guarantee power-loss reliability.  As a result, it
queries the hardware to see if an SC_SYNC operation is supported.  If
the hardware says that it is not, then the filesystem will not mount
read-write, since it can't guarantee that when it asks for data to be
flushed, the hardware will comply. Some CF cards do work, and some
don't.  Some lie and say that the support the command when they really
don't.

If you want to avoid this check, pass "-o sync=optional" to the mount
command, and the filesystem will mount read-write, even if the hardware
doesn't claim to support SC_SYNC.

David



> -----Original Message-----
> From: Robert Murrell [mailto:community-noreply@qnx.com]
> Sent: August-12-11 11:28 AM
> To: general-filesystems
> Subject: Re: RE: Partial files being create?
> 
> Great.  This isn't a problem under FAT file systems, other than lost
> clusters which are easily recovered.
> 
> Now for the next questions:
> 
> Does the QNX 6 filesystem need a new IPL?  We are using a custom ipl-
> diskpc2.
> 
> It looks like I use the devctl function to programmatically do what
> chattr is doing.  Is there any guidance on how to set/clear this hold
> flag using devctl?
> 
> The section on "Required properties of the device" is unclear.  Will
> the QNX 6 filesystem work with a Compact Flash card?
> 
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post88076
Robert Murrell
08/12/2011 1:35 PM
post88082
Re: RE: RE: Partial files being create?   
This just keeps getting better.  It took me months to get the source code for ipl-diskpc2.  Before I turn the rudder to 
the Titanic and switch to QNX 6, I need to understand the power-fail errors that the QNX 4 file system is susceptible to
.  I can work around lost data and partial files.  Are there any potential errors that can't be fixed by chkfsys?
David Sarrazin
08/12/2011 2:35 PM
post88084
RE: RE: RE: Partial files being create?   
If running chkfsys is okay, then you should be good.  I'm trying to
think of cases that chkfsys can't repair, but nothing comes to mind.  If
you can handle truncated files, then you should be okay.  Your original
post made it sound like you couldn't tolerate any form of corruption to
the /.boot or /.altboot files.  QNX6 filesystem is still the best way to
guarantee that the file is either 100% the old content, or 100% the new
content, but if you can handle the partial write case, then keep what
you have.

> -----Original Message-----
> From: Robert Murrell [mailto:community-noreply@qnx.com]
> Sent: August-12-11 1:35 PM
> To: general-filesystems
> Subject: Re: RE: RE: Partial files being create?
> 
> This just keeps getting better.  It took me months to get the source
> code for ipl-diskpc2.  Before I turn the rudder to the Titanic and
> switch to QNX 6, I need to understand the power-fail errors that the
> QNX 4 file system is susceptible to.  I can work around lost data and
> partial files.  Are there any potential errors that can't be fixed by
> chkfsys?
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post88082
Robert Murrell
08/12/2011 2:48 PM
post88085
Re: RE: RE: RE: Partial files being create?   
We will likely switch to the QNX 6 file system if we can be assured that we can get Compact Flash cards that support 
thei SC_SYNC operation.  If we have to stay with QNX 4, I can increase the reliability by not writing the image header 
until the last block is written to the file.  The secondary boot loader will detect this condition and can accommodate 
it.
Robert Murrell
08/12/2011 3:10 PM
post88086
Re: RE: RE: RE: Partial files being create?   
Is there a generic or standard terminology for SC_SYNC?  What property should I be asking for from Compact Flash 
manufacturers?
David Sarrazin
08/12/2011 3:17 PM
post88087
RE: RE: RE: RE: Partial files being create?   
From the driver expert:

It is translated to an IDE Flush Cache command (0xe7 or 0xea).

Most of the commands from the filesystem are using SCSI CAM terminology.

> -----Original Message-----
> From: Robert Murrell [mailto:community-noreply@qnx.com]
> Sent: August-12-11 3:10 PM
> To: general-filesystems
> Subject: Re: RE: RE: RE: Partial files being create?
> 
> Is there a generic or standard terminology for SC_SYNC?  What property
> should I be asking for from Compact Flash manufacturers?
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post88086
Robert Murrell
08/12/2011 3:42 PM
post88089
Re: RE: RE: Partial files being create?   
Where do I get the secondary boot loader ipl-diskpc2-fsq6?  It doesn't come with the QNX 6.5.0 distribution.  Only ipl-
diskpc2 and ipl-diskpc2-flop are provided.
Robert Murrell
08/12/2011 4:53 PM
post88091
Re: RE: RE: Partial files being create?   
The system seems to boot with ipl-diskpc2, but its coming up read-only with Compact Flash cards I have here.

Return


  A subsidiary of BlackBerry             All content ©2004-2013, QNX Software Systems