Why large USB SATA drives suck!

For the second time in the last 6 months I’ve got forensic evidence that not only does silent data corruption happen in real life, but also that “green” drives, hidden inside external USB enclosures, seem to be at higher risk than native attached SATA. I’ve seen this with two different enclosures, with different SATA drives, connected to different servers running different OS versions.

For a while now I’ve been using ZFS, first with OpenSolaris and then with Nexenta. Recently I mirrored a significant amount of data from my internal drives on my Nexenta file server to external USB, in preparation for migrating the data to a new server with new drives at home.

My mistake was not running a ZFS scrub after mirroring the USB drives with the internal zpools.

As it turns out, there were significant numbers of files (MP3s and AVIs mostly) that were corrupt on the USB copy. How do I know? ZFS uses a checksum on every block written, and it validates that checksum on every read. There is overhead associated with this, but it’s great to know that the filesystem is protecting your data.

This is not the first time that ZFS has told me that data corruption has occurred. In both cases, the data was stored on USB connected drives, and in both cases, Western Digital “green” drives were in the external enclosure.

I don’t know if it was the USB cable, the USB-SATA Controller, or the WD Green drives that were at fault. What I do know is that those of you using non-checksumming filesystems may have data corruption on your external drives, and you won’t know it until it’s too late.