First off I’d like to apologize for the lack of updates lately, I’ve been dealing with some dire problems on my file server.  Good news is however all my data is safe so, I’ll be able to focus again without that frighting thought in the back of my head that my data is going to disappear.

In light of my issues, I started researching the latest and greatest innovations in regards to file and volume management.  Which brings me to ZFS…

ZFS is probably one of the greatest things available from the open source community in years, possibly the last decade!  Hell, I’m not even sure where to start telling you about this beauty.  So I apologize in advance if I start to rant and jump all over the place.

ZFS is basically a file system and a logical volume manager.  Where as file systems like EXT2,3 and 4 are simply file systems with “some” features for data management.  ZFS is a technology developed by Sun Microsystems, the same folks who gave us Java.

In short here are some of the features to get your juices flowing.

  • High Storage Capacity, 16Exabytes.  To give a glimpse at how big that is, Globally there are approximately 21 Exabytes of bandwidth consumed in one calendar month.
  • Volume Management built into the file system, more details further on.
  • Granular File System Snapshots.  Think VM snapshots and system restore, but built into the file system.  Volume wide, or down to a single file level.  Imagine before recompiling your kernel with some scary development features, snapshot the file system and if all goes wrong you’re back to your original safe configuration.
  • Copy on write features.
  • Automatic data integrity checking, aka scrubbing on the fly.
  • RAID Support, aka raidz.  Striping, Mirroring. Single, Double and Triple Parity (aka raid 5, 6)
  • Granular Compression.  Entire volumes, down to the folder level can have different levels of compression, even using gzip1-9 if you wish.
  • Instant file system creation.  No more days of waiting for that magnificent RAID5 array to build.  raidz is built, formatted and mounted within a seconds.
  • Cache Management, you can optionally add a few solid state drives for lightning fast access to commonly accessed data.  Think hard drive cache.  Some drives sold these days have 64 megs of cache.  Now think about having several terabytes of cache for your high density raid array!
  • And the last one I’ll mention here which is my absolute favorite. On the fly deduplication.  Say you have a collection of MP3′s collected over the past 10 years.  Chances are organization can get a little bit messy over time.  Duplicate files start to become a big issue as well, consuming more and more space as the collection grows.  ZFS instantly solves this with deduplication, but not just at the file level.  ZFS uses a 256SHA hash,  so even if a file is similar but not exactly the same it can yank out the duplicate portion.  All while still maintaining your original files and structure.

Ok, lots of reading there.  ZFS in a nutshell is your administrator’s wet dream of  a file system.  It’s so adaptable and user friendly, it doesn’t even take weeks of training and certification to use.

Now you might be asking yourself what do I have to do to get this!  Well, there in lies the catch at the moment.  ZFS is infact open source, but there are apparently conflicts within GPL licenses.  But don’t curse the GPL just yet.

OS X had and ditched support for ZFS.  (*Good move Apple, way to shoot yourself in the foot), BSD already supports it (limited support, not full feature set… for now!)  and Linux projects have been gaining steam.  FreeNAS already has limited support for it, upto v6 I believe.  But for the real good news, Nexenta a similar product to FreeNAS but built on and around ZFS currently has nearly full support for ZFS.  v21 is where they are at the moment as far as I know.

Over the next couple of months I’m going to be putting together some tutorials using ZFS that are pretty much going to make you tear your whole fileserver apart and rebuild it from scratch to use ZFS.

That’s all for now, just wanted to give some insight on whats been happening.

Tags: