Flash: A Little ZFS History

Flash: A Little ZFS History

Adam Leventhal had been working for years at Sun with their Fishworks team, which leveraged a new piece of hardware referred to as Thumper, combined with Solaris 10's ZFS. He is no longer with Sun, but still has some great history on his personal blog with ZFS and acceleration.


Read Optimized Flash

Flash normally comes in two different flavors, Read Optimized Flash, where it is cheap & fast, but not so reliable. When caching LOTS of information, to reduce read access to rotating rust, the benefits are extremely beneficial since random access time will drop on large storage pool on such monster storage platforms like Sun's original Thumper, pictured above.

The Adaptive Read Cache in ZFS was designed to be extended. Disk are slow, DRAM is expensive, Flash meets a nice niche in the middle. Flash has limited write cycles, but if it burns out in a cache, it is no big deal since the cache-miss would just go to the disk.

A lot of businesses have been talking about replacing hard drives with Flash, but their long term storage is not as secure. Flash is better used as cache. Sun had affectionately called their Read Cache technology as "Readzilla" when it is applied to ZFS.



Write Optimized Flash

Another area of pain experienced by uses is with write bottlenecks. The more writing you do, the more random access to the disks may occur, and the more latency is produced because of the seek time as the mechanical heads move slowly across the platters.

Taking writes, turning them into sequential writes, is a big help in modern file systems like ZFS. If one could take the writes and commit them to another place, where there are no mechanical steppers, further advances is speed can be accomplished. This is where Sun came up with "Logzilla" - using Write Optimized Flash to accelerate this process.

ZFS has a feature where one can place their writes on dedicated infrastructure and Flash designed to handle writes quickly yet reliably is extremely beneficial. This is a much more expensive solution than disk, but because it is faster and non-volatile, a system crash when a write is being committed to disk will not be lost as it would be in straight DRAM.



Non-Volatile DRAM

Adam mentioned non-volatile DRAM as an option in his personal blog entry (as well as defunct Sun blog entry.) Get a UPS and plug in the NV-DRAM card, to get the benefits of DRAM speed, non-volatility of Flash, and virtually unlimited writes... this seems like a winner.

What no one tells you is that your UPS becomes a more critical component than ever before. If you do not replace your batteries in time (no generator and a truck hits a poll) - your critical data might be lost in an hour.

Network Management

Nearly all network management shops deal with high quantities of reads and writes on a steady load... with A LOT of head-stepping. This comes from the need to poll millions of distinct data every minute, roll the data up, stick it in a database, and roll the data to keep it trim.

For environments like this, ZFS under Solaris is optimal, leveraging Read and Write optimized Flash. In a clustered environment, it may become important to keep these write optimized flash units external, on centralized infrastructure.

If performance management in network management is your life: Solaris ZFS is your future with Readzilla and Logzilla. Nothing out there compares from any other Operating System for the past half-decade.