[tlug] Help with fsck and ocfs2 (or even ext4?)...

[Raymond Wan <rwan.kyoto@example.com>]

Hi all,

I'm having a problem with a file system at work and I'm unfortunately
not very good with these type of problems.

The file system is a Oracle Cluster Filesystem (ocfs2) on a SAN.
While performing a copy, one of the servers attached to it was frozen
so I thought I should restart it.  Well, not surprisingly, the file
system didn't like it.

Upon restarting, it was mounted as read only.  I ran fsck.ocfs2 and
many lost files were found.  But I ran it again to make sure it was
fine and this error came up:

...
Pass 1: Checking inodes and blocks
[Scanning inodes 100%]
  I/O read disk/cache: 1000MB / 392MB, write: 0MB, rate: 2.10MB/s
  Times real: 11m57.073s, user: 4m1.084s, sys: 0m1.030s
Pass 2: Checking directory entries
pass2: Bad magic number in directory block while reading dir block 1439634968
pass2: Bad magic number in directory block while reading dir block 1439634969
pass2: Bad magic number in directory block while reading dir block 1439634970
  I/O read disk/cache: 16MB / 2239MB, write: 0MB, rate: 1.48MB/s
  Times real: 0m11.971s, user: 0m1.156s, sys: 0m0.041s
Pass 3: Checking directory connectivity
[DIR_DOTDOT] Directory inode 4273782381 is referenced by a dirent in
directory  4273782380 but its '..' entry points to inode 0. Fix the
'..' entry to reference 4273782380? <y> y
 fix_dot_dot: Bad magic number in directory block while iterating
through dir inode 4273782380's directory entries.
 I/O read disk/cache: 0MB / 1MB, write: 0MB, rate: 0.00MB/s
 ...
 All passes succeeded

No matter how many times I run the command, I get these same errors.
There is a debugfs.ocfs2 command which I can run interactively and
figure out that 4273782381 isn't a file I need.  But I don't know if
this command can fix my problem -- I've never used it or the ext4
equivalent.

When I mount it, it now mounts as read-write but once I enter the
direction in question, the file system switches to read-only.  There
isn't a whole lot of information for ocfs2.  I tried to register
myself on the ocfs2 mailing list and it seems they're not approving my
application.  I also tried ServerFault with no luck.

I expanded my search to see what steps are there for ext4 and what
I've seen so far is "keep running fsck until it's happy".  And in the
end, if you can't make fsck happy, then you copy the files out and
reformat...  I'm unable to find another solution.  I'm happy to erase
the problematic files, actually...  Anything to save the other
files...

Anyway, any suggestions would be appreciated!  I am considering
copying the files out and reformatting as a last resort, but at 70 TB,
I would rather not...  :-(

On an unrelated note, does anyone have an opinion about GFS2 and if
it's better than ocfs2?

Thank you!

Ray