> la où le ext3, ext2 et reiserFS pouvaient aller se réhabiller (perte de /usr, données sensibles, etc...) ...
Bon troll. Au moins pour ext3.
Trouvé sur mailing Fedora. Un mec a fait plein de tests avec différents systèmes de fichier. http://www.redhat.com/archives/fedora-test-list/2004-July/msg00034.(...) I finally managed to turn off the write caching on the SATA drives (see
patch below) and I have re-run the tests on all four journaling file
systems. And again I am completely stumped by the results.
The test is really pretty simple. It is hooked to a machine that cycles
the power. It runs for 5 minutes and then the power is turned off for
1 minute (to simulate the plug being pulled). For this last set of tests
it has 3 hard drives connected, one Parallel IDE and two Serial ATA. The
system boots a minimal install of Fedora Core 2 from the IDE drive. No
tests are running on the IDE drive. In the rc.local file it starts the
fsstress test http://ltp.sf.net/nfs/fsstress.tgz(...) and the three scripts
below (to simulate writing into a log file) on each of the SATA drives.
The ext3 have almost a perfect record with the write cache off: I have
run over 300 cycles on the two drives and only had two corrupted lines
in the output files. So out of 600 total cycles on the two drives there
were only two lines with bad data, I think that is a pretty good record.
None of the other journaling file systems have come anywhere near this
performance. After 3 or 4 power cycles, ReiserFS became corrupted to
the point that the system would not boot up (the fsck failed and the
bootup stopped there). XFS never got corrupted to the point it wouldn't
boot, but with approximately 100 power cycles on each drive, one drive
had 73 corrupted lines and the other had 82. With JFS after 15 power
cycles one of the drives was corrupted and the system would no longer
boot up (fsck failed again).
I just can't understand what is happening, it makes no sense to me that
one file system would be almost perfect and three would fail so
dramatically. I am going to re-run the tests on all 4 file systems to
verify that it is repeatable.
Should I report these problems to the upstream projects (Reiser, XFS, JFS)?
Réponse/éclaississement d'Alan Cox de ses petites pertes de donnée avec ext3 qui sont "normals" : [...]
Unless you are doing data journalling or some kind of userspace
transactions you wouldn't expect file contents to be perfect. Data
journalling has a big performance cost.
[...]
Your expectations seem at odds with what journalling provides. A journalled
fs can be recovered by log replay. It doesn't guarantee that user data is
recovered precisely. It guarantees that user data is recovered to those
points where it was committed.
Thus
open file O_APPEND
write stuff
close it
repeat. doesn't guarantee "stuff" will always be committed - it just
guarantees that the fs will be structurally sound
open file O_APPEND
write stuff
fsync
close
OTOH says that after the fsync has returned you can be sure the data
just wrote before it *will* still be there.
Ext3 data journalling journals everything which is a bit slower but can
be appropriate for some applications (and actually for big NFS servers
often turns out to be faster because of the NFS commit behaviour)
Réponse de Arjan van de Ven : this is expected behavior; by default ext3 has a journaling mode that is
more safe than defaults of other filesystems, eg when you create a new
file or extend an existing one, it will not commit the new size to disk
before the data has been sent to the disk. Other journaling filesystems
do this entirely asynchronous.
The cost is raw performance, which is why ext3 has a mount option to
emulate the behavior of the other journaling filesystems in both speed
and, ehm, data security.
Your investigation proves that we default to the right mode ;)
Donc : il faut désactiver l'écriture en arrière plan du disque avant de faire des tests !
Sinon ça ne marche pas et c'est un problème connu ! Sauf pour certains disques SCSI.
[^] # Re: JFS
Posté par 007 . En réponse au sondage Le système de fichiers que j'utilise le plus :. Évalué à 4.
Bon troll. Au moins pour ext3.
Trouvé sur mailing Fedora. Un mec a fait plein de tests avec différents systèmes de fichier.
http://www.redhat.com/archives/fedora-test-list/2004-July/msg00034.(...)
I finally managed to turn off the write caching on the SATA drives (see
patch below) and I have re-run the tests on all four journaling file
systems. And again I am completely stumped by the results.
The test is really pretty simple. It is hooked to a machine that cycles
the power. It runs for 5 minutes and then the power is turned off for
1 minute (to simulate the plug being pulled). For this last set of tests
it has 3 hard drives connected, one Parallel IDE and two Serial ATA. The
system boots a minimal install of Fedora Core 2 from the IDE drive. No
tests are running on the IDE drive. In the rc.local file it starts the
fsstress test http://ltp.sf.net/nfs/fsstress.tgz(...) and the three scripts
below (to simulate writing into a log file) on each of the SATA drives.
The ext3 have almost a perfect record with the write cache off: I have
run over 300 cycles on the two drives and only had two corrupted lines
in the output files. So out of 600 total cycles on the two drives there
were only two lines with bad data, I think that is a pretty good record.
None of the other journaling file systems have come anywhere near this
performance. After 3 or 4 power cycles, ReiserFS became corrupted to
the point that the system would not boot up (the fsck failed and the
bootup stopped there). XFS never got corrupted to the point it wouldn't
boot, but with approximately 100 power cycles on each drive, one drive
had 73 corrupted lines and the other had 82. With JFS after 15 power
cycles one of the drives was corrupted and the system would no longer
boot up (fsck failed again).
I just can't understand what is happening, it makes no sense to me that
one file system would be almost perfect and three would fail so
dramatically. I am going to re-run the tests on all 4 file systems to
verify that it is repeatable.
Should I report these problems to the upstream projects (Reiser, XFS, JFS)?
Réponse/éclaississement d'Alan Cox de ses petites pertes de donnée avec ext3 qui sont "normals" :
[...]
Unless you are doing data journalling or some kind of userspace
transactions you wouldn't expect file contents to be perfect. Data
journalling has a big performance cost.
[...]
Your expectations seem at odds with what journalling provides. A journalled
fs can be recovered by log replay. It doesn't guarantee that user data is
recovered precisely. It guarantees that user data is recovered to those
points where it was committed.
Thus
open file O_APPEND
write stuff
close it
repeat. doesn't guarantee "stuff" will always be committed - it just
guarantees that the fs will be structurally sound
open file O_APPEND
write stuff
fsync
close
OTOH says that after the fsync has returned you can be sure the data
just wrote before it *will* still be there.
Ext3 data journalling journals everything which is a bit slower but can
be appropriate for some applications (and actually for big NFS servers
often turns out to be faster because of the NFS commit behaviour)
Réponse de Arjan van de Ven :
this is expected behavior; by default ext3 has a journaling mode that is
more safe than defaults of other filesystems, eg when you create a new
file or extend an existing one, it will not commit the new size to disk
before the data has been sent to the disk. Other journaling filesystems
do this entirely asynchronous.
The cost is raw performance, which is why ext3 has a mount option to
emulate the behavior of the other journaling filesystems in both speed
and, ehm, data security.
Your investigation proves that we default to the right mode ;)
Informations supplémentaires et utiles données par Bill Rugolsky Jr. :
http://www.redhat.com/archives/fedora-test-list/2004-July/msg00036.(...)
Il y a aussi eu un très long thread en avril :
http://www.redhat.com/archives/fedora-test-list/2004-April/msg02646(...)
Donc : il faut désactiver l'écriture en arrière plan du disque avant de faire des tests !
Sinon ça ne marche pas et c'est un problème connu ! Sauf pour certains disques SCSI.