Welcome to ZFS! | The Observation Deck

If you haven’t already seen it, ZFS is now available for download, marking a major milestone in the history of filesystems. Today is a way station down a long road: for as long as I have known Jeff Bonwick, he has wanted to solve the filesystem problem – and about five years ago, Jeff set out to do just that, starting (as Jeff is wont to do) from a blank sheet of paper. I vividly remember Jeff describing some of his nascent ideas on my whiteboard; the ideas were radical and revolutionary, their implications manifold. I remember thinking “he’s either missed something basic that somehow invalidates these ideas – or this is the most important development in storage since RAID.” As I recently recounted, Jeff is the reason that I came to Sun almost a decade ago – and in particular, I was drawn by Jeff’s contagious belief that nothing is impossible simply because it hasn’t been done before. So I knew better than to doubt him at the time – and I knew that the road ahead promised excitement if nothing else. Years after that moment, there is no other conclusion left to be had: ZFS is the most important revolution in storage software in two decades – and may be the most important idea since the filesystem itself. That may seem a heady claim, but keep reading…

To get an idea of what ZFS can do, first check out Dan Price’s awesome ZFS flash demo Then join me on a tour of today’s ZFS blog entries, as ZFS developers and users inside Sun illustrate the power of ZFS: ease of administration, absolute reliability and rippin’ performance.

Administration. If you’re an administrator, start your ZFS blog tour with Martin Englund‘s entry on ZFS from a sysadmin’s view. Martin walks you through the ease of setting up ZFS; there are no hidden wires – it really is that easy! And if, as a self-defence mechanism, your brain refuses to let you recall the slog of traditional volume management, check out Tim Foster‘s entry comparing ZFS management to Veritas management. (And have we mentioned the price?) For insight into the design principles that guided the development of the administration tools, check out Eric Schrock‘s entry on the principles of the ZFS CLI. Eric’s entry and his design reflect the principles that we used in DTrace as well: make it simple to do simple things and make it possible to do complicated things. As you can imagine, this simplicity of management is winning fans both inside and outside of Sun. For some testimonials, check out Lin Ling‘s entry on the love for ZFS – both Lin’s and our Beta customers’. As Lin’s entry implies, a common theme among ZFS users is “if I only had this when…”; check out James McPhearson‘s entry wishing he had ZFS back in the day.

And if you think that the management of ZFS couldn’t get any easier, check out Steve Talley‘s entry on managing ZFS from your browser. Steve’s work highlights the proper role for GUI admin tools in a system: they should make something that’s already simple even simpler. They should not be used to smear lipstick over a hideously over-complicated system – doing so leads to an unresolvable rift between what the tool is telling you the system looks like, and what the system actually looks like. Thanks to the simplicity of ZFS itself, there is no second guessing about what the GUI is actually doing under the hood – it’s all just gravy!

Speaking of gravy, check out the confluence of ZFS with another revolutionary Solaris technology in Dan Price‘s entry on ZFS and Zones thanks to some great integration work,local zone administrators can have the full power of ZFS without compromising the security of the system!

For details on particular features of ZFS, check out Mark Maybee‘s entry on quotas and reservations in ZFS. Unlike some other systems, quotas and reservations are first-class citizen iin ZFS, not bolted-on afterthoughts. Die, /usr/sbin/quota, die! And for details on another feature of ZFS, check out Mark Shellenbaum‘s entry on access control lists in ZFS, and Lisa Week‘s entry describing why ZFS adopted the NFSv4 ACL model. Like quotas and reservations, ACLs were a part of the design of ZFS – not something that was carpet-bombed over the source after the fact.
Reliability. Unlike virtually every other filesystem that has come before it, ZFS is designed around unreliable hardware. This design-center means that ZFS can detect – and correct! – errors that other filesystems just silently propagate to the user. To get a visceral feel for this, read Eric Lowe’s entry on ZFS saving the day. Reading this entry will send a chill up your spine: Eric had a data-corrupting hardware problem that he didn’t know he had until ZFS. How much data is being corrupted out there today because pre-ZFS filesystems are too trusting of faulty hardware? More to the point, how much of your data is being corrupted today? Yeah – scary, ain’t it? And not only can ZFS detect hardware errors, in a mirrored configuration it can correct them. Fortunately, you don’t have to have busted hardware to see this: look at Tim Cook’s entry demonstrating ZFS’s self-healing by using dd to simulate date corruption.But if problems like Eric’s are all over the place, how is anyone’s data ever correct? The answer is pretty simple, if expensive: you pay for reliability by buying over-priced hardware. That is, we’ve compensated for dumb software by having smart (and expensive) hardware. ZFS flips the economics on its head: smart software allows for stupid (and cheap) hardware – with ultra-high reliability. This is a profound shift; for more details on it check out Richard Elling‘s entry on the reliability of ZFS.ZFS is reliable by its architecture, but what of the implementation? As Bill Moore writes, testing ZFS was every bit as important as writing it. And testing ZFS involved many people, as Jim Walker describes in his entry on the scope of the ZFS testing effort.
Performance. So fine: ZFS is a snap to administer, and it’s ultra-reliable – but at what performance cost? The short answer is: none, really – and in fact, on many workloads, it rips. How can you have such features and still have great performance? Generally speaking, ZFS is able to deliver great performance because it has more context, a phenomenon that Bill Sommerfeld notes is a consequence of the end-to-end principle. To see how this unlocks performance, look at Bill Moore’s entry on I/O scheduling; as Bill describes (and as I can personally attest to) ZFS is much smarter about how it uses I/O devices than previous filesystems. For another architectural feature for performance, look at Neil Perrin’s entry on the ZFS intent log – and chase it with Neelakanth Nadgir‘s entry taking you through the ZIL code.If you’re looking for some performance numbers, check out Roch Bourbonnais’s entry comparing the performance of ZFS and UFS. Or let Eric Kustarz take you to school, as you go to Filesystems Performance 101: Disk Bandwidth, Filesystems Performance 102: Filesystem Bandwidth and finally graduate to Filesystems Performance 201: When ZFS Attacks!

So given that ZFS is all that, when can we start forgetting about every other on-disk filesystem? For that, we’ll need to be able to boot off ZFS. Bad news: this is hard. Good news: Tabriz Leman and the rest of the ZFS Boot team are making great progress, as Tabriz describes in her entry on booting ZFS. Once we can boot ZFS – that is, once we can assume ZFS – all sorts of cool things become possible, as Bart Smaalders brainstorms in his entry on the impact of ZFS on Solaris. As Bart says, this is just the beginning of the ZFS revolution…

Finally, this has been a long, hard slog for the ZFS team. Anyone who has worked through “crunch time” on a big project will see something of themselves in Noel Dellofano‘s entry on the final push. And any parent can empathize with Sherry Moore‘s entry congratulating the team – and looking forward to having her husband once again available to help with the kids. So congratulations to everyone on the ZFS team (and your families!) – and for everyone else, welcome to ZFS!

Technorati tags: OpenSolaris Solaris ZFS