Flash: A Little ZFS History

Flash: A Little ZFS History

Adam Leventhal had been working for years at Sun with their Fishworks team, which leveraged a new piece of hardware referred to as Thumper, combined with Solaris 10's ZFS. He is no longer with Sun, but still has some great history on his personal blog with ZFS and acceleration.


Read Optimized Flash

Flash normally comes in two different flavors, Read Optimized Flash, where it is cheap & fast, but not so reliable. When caching LOTS of information, to reduce read access to rotating rust, the benefits are extremely beneficial since random access time will drop on large storage pool on such monster storage platforms like Sun's original Thumper, pictured above.

The Adaptive Read Cache in ZFS was designed to be extended. Disk are slow, DRAM is expensive, Flash meets a nice niche in the middle. Flash has limited write cycles, but if it burns out in a cache, it is no big deal since the cache-miss would just go to the disk.

A lot of businesses have been talking about replacing hard drives with Flash, but their long term storage is not as secure. Flash is better used as cache. Sun had affectionately called their Read Cache technology as "Readzilla" when it is applied to ZFS.



Write Optimized Flash

Another area of pain experienced by uses is with write bottlenecks. The more writing you do, the more random access to the disks may occur, and the more latency is produced because of the seek time as the mechanical heads move slowly across the platters.

Taking writes, turning them into sequential writes, is a big help in modern file systems like ZFS. If one could take the writes and commit them to another place, where there are no mechanical steppers, further advances is speed can be accomplished. This is where Sun came up with "Logzilla" - using Write Optimized Flash to accelerate this process.

ZFS has a feature where one can place their writes on dedicated infrastructure and Flash designed to handle writes quickly yet reliably is extremely beneficial. This is a much more expensive solution than disk, but because it is faster and non-volatile, a system crash when a write is being committed to disk will not be lost as it would be in straight DRAM.



Non-Volatile DRAM

Adam mentioned non-volatile DRAM as an option in his personal blog entry (as well as defunct Sun blog entry.) Get a UPS and plug in the NV-DRAM card, to get the benefits of DRAM speed, non-volatility of Flash, and virtually unlimited writes... this seems like a winner.

What no one tells you is that your UPS becomes a more critical component than ever before. If you do not replace your batteries in time (no generator and a truck hits a poll) - your critical data might be lost in an hour.

Network Management

Nearly all network management shops deal with high quantities of reads and writes on a steady load... with A LOT of head-stepping. This comes from the need to poll millions of distinct data every minute, roll the data up, stick it in a database, and roll the data to keep it trim.

For environments like this, ZFS under Solaris is optimal, leveraging Read and Write optimized Flash. In a clustered environment, it may become important to keep these write optimized flash units external, on centralized infrastructure.

If performance management in network management is your life: Solaris ZFS is your future with Readzilla and Logzilla. Nothing out there compares from any other Operating System for the past half-decade.

McAfee: Purchased by Intel After Fiasco

McAfee: Purchased by Intel After Fiasco

What is Virus Protection?

Operating Systems like Microsoft Windows offers mechanisms to install software automatically. Sometimes, it the mechanism is a bug. Other times, it is a key which may have been purchased or hacked, and later leveraged to deposit viruses or spy-ware. Sometimes, the OS just offers too much freedom to the user, to allow them to install anything they would like (anywhere they would like), and when they unwittingly install a piece of software on purpose, the machine becomes infected.

Some consider it the computing system equivalent to the Mafia, "You want to be safe, pay us some protection money, and you'll be safe." They work to make the computing environment more rigid because the operating system vendor (in this case, Microsoft) was too lazy.

To make the environment more rigid, inspection is done for known snips of code on files loaded on the hard drive, coming in or out via email, or through tools like web browsers. These pieces of code that are searched for, basically subets of the possible virus or worms which can be used to identify them, are called "signatures".

The "signatures" are distributed from central locations from the Mafia's God Father to the software applications which some people choose to install on their computer, hereafter referred to as the Hitman. It is the job of the Hitman on your computer to whack the virus... or regularly encourage you to pay-up if you did not pay your security bill.

Leading up to the Acquisition:

Less than half a year ago, McAfee distributed a virus signature update that identified a core Microsoft Windows file a problem and whacked it.

McAfee update crippled some Windows PCs by quarantining or deleting a file crucial to Windows operation, called “svchost.dll”.

The bug, McAfee said, meant that “less than half of one per cent” of business customers, and a smaller number of consumer customers, could not use their computers. The company did not release any detailed figures, but said that the problem nly occurred on machine running Windows XP Service Pack 3 in combination with a specific build of McAfee’s antivirus product.

Large segments of society, especially emergency services who were unfortunate enough to pick Microsoft Windows for their core infrastructure, in combination with McAfee, and various service packs were scrambling for cover.

Reported victims include Kansas City Police Department and and the University of Kansas Hospital and about a third of the hospitals in Rhode Island. PCs also went haywire at Intel, the New York Times reports, citing Twitter updates from workers at the chip giant as a source.

First hand experiences from an Iowa community emergency response centre, ironically running a disaster recovery exercise at the time, can be found in a posting to the Internet Storm Centre here. The Register has heard from a senior security officer at a net infrastructure firm that was also hard hit by the snafu, as reported in our earlier story here.

McAfee picked up very bad reputation after this event.

To be fair, a virus signature is nothing more than a pattern of bits that can appear in a file at a particular set of locations, so it is amazing that after all these years, with so many virus signature creators, that this has not happened earlier.

The New God Father:

After a very bad spring and summer, the cost of the help desk support to repair all of those old machines, lost customers who would stop using their products, bad media coverage from the mishap, and new customers who were not very interested in taking a chance on them - someone else was really needed to clean up their reputation.

Intel Corporation made purchased McAfee. What surprises me is that fewer media outlets had connected the purchase with the recent virus signature failure.

Network Management Connection:

Microsoft Windows systems are tremendous targets for viruses and worms. With Network Management systems which must be located in a DMZ and connect to millions of potential end points, such platforms should be considered a virus & worm distribution system, and avoided.

The application of virus definitions to such production systems can disrupt the reputation of a third party management company and put them out of business, the same way McAfee tarnished reputation needed to be consumed by Intel.

Linux: 5 Year Old Root Exploit Finally Patched

Security Focus:
It has been a over half decade, but a Linux kernel root exploit has finally been patched. Yes, Oracle Enterprise Linux, RedHat, and others have been running around with this issue for a long time.


For your Novell fans, the SUSE distribution has been OK since 2004, but it has not trickled down to the other distributions since the fix had not been incorporated into the official kernel until now.

Network Management:
In a world of network management where a central or even distributed systems monitor or manage millions of potential device across many thousands of networks, a root exploit in an operating system kernel dating back over a half decade is extremely high risk.

If it has to run and has to run securely - a generic Linux distribution may not fit the bill.

Look for Operating System vendors who have a strong record with understanding Data Centers and managing networks, not just OS vendors who can do it more cheaply.

SunRay: VDI Out of This World!

SunRay: VDI Out of This World!

The SunRay Thin Clients can be used anywhere, even off the face of the earth, while traveling in airplanes!

This snapshot of the thin client functioning on an airplane tray is pretty amazing!

Solaris Express Resurrected and Other Changes

Solaris Express Resurrected
For those of you who have been following Solaris for a long time, it had started as an open source operating system, based upon BSD, merged with SVR4, Sun started the process of a re-write to open-source it again with the OpenSolaris project, controversially started an binary distribution also called OpenSolaris, and canceled Solaris Express program.

With the purchase of Sun by Oracle, the direction of the wind has changed. Solaris Express has been revived.

We will have a Solaris 11 binary distribution, called Solaris 11 Express, that will have a free developer RTU license, and an optional support plan.

Solaris 11 Express will debut by the end of this calendar year, and we will issue updates to it, leading to the full release of Solaris 11 in 2011.

Sad Changes
On the sad side, the real-time feed of source code changes to the community will end, and binary distributions of OpenSolaris will also end.

We will not release any other binary distributions, such as nightly or bi-weekly builds of Solaris binaries, or an OpenSolaris 2010.05 or later distribution.

Status Quo
On the bright side, source code will continue to be published, so existing down-stream OpenSource projects will be able to leverage it, much the same way Oracle works with RedHat's distribution of Linux today.

We will distribute updates to approved CDDL or other open source-licensed code following full releases of our enterprise Solaris operating system.

Also, upstream contributions will continue to be accepted.

We will continue active open development, including upstream contributions, in specific areas that accelerate our overall Solaris goals.

Examples include our activities around Gnome and X11, IPS packaging, and our work to optimize ecosystems like Apache, OpenSSL, and Perl on Solaris.

Acceptance as peers into the Solaris community (full access to real-time source code and contributions) will occur on case-by-case basis. This is really no difference from the existing community - not any yahoo off the internet could make changes to the source code.

We will have a technology partner program to permit our industry partners full access to the in-development Solaris source code through the Oracle Technology Network (OTN).

This will include both early access to code and binaries, as well as contributions to us where that is appropriate.

The landscape has changing, things are returning to more of the way Sun used to be prior Jonathan Schwartz. It looks almost like a hybrid approach between Scott McNealy and Jonathyn on the software front.

Differences
Honestly, there is not that much of a difference, except competitors will not have access to new source code of Solaris as quickly to facilitate their copying. This has the potential to make Solaris a stronger competitor in the marketplace.

Instead of OpenSolaris competing with other binary distributions, it appears to become more of an example "gold disk"with mosts of the bugs worked out, with the source code for other derived contributions to base their work upon. If people are really serious about contributing, they still can, through a different web site.

Let's see how this hybrid development model works!

Oracle-Sun 08/2010 Systems Strategy

Oracle-Sun 08/2010 Systems Strategy

Abstract:
John Fowler, Executive Vice President, had a web conference concerning the systems strategy for the merged Oracle & Sun company. The full PDF of slides was made available. For those who are more interested in systems oriented news, captured [high resolution] screen shots are captured with the interesting highlights.

Solaris Roadmap:
Solaris 11 is coming next year, with multiple upgrades scheduled for the next 5 years.

Solaris Features:
Oracle understands what Solaris is - this is the operating system where if the data needs to be secure and the business must function, this is the advocated direction for the business.

SPARC Trajectory:
Systems based upon the open SPARC central processors with the Solaris operating system will continue to receive upgrades to deliver performance improvements over the next 5 years.

SPARC Roadmap:
A hardware announcement is scheduled to happen later this year and the roadmap shows a doubling of perforance before the end of the year.

SPARC Direction:
One SPARC architecture, one operating system, one systems management, one virtualization strategy, more memory, more threads, more cores, and system aggregate throughput increases of 2x every 2 years.

Storage Trajectory:
Substantial "order of magnitude" improvements over the net 5 years.

Oracle Storage Roadmap:
Industry leading now, will continue to do so into the future.

Oracle Storage:
Oracle is number 1 or first to market on an increasing number of technologies.

Conclusion:
Oracle is the one to trust and the one who will deliver the goods for the next 5 years!

It will shortly be a very good time to build out a Network Operations Center, to get the most advanced and stable operating system on the latest next generation SPARC architecture.

Solaris 11: 2011 Confirmed

Solaris 11: 2011 Confirmed


The Concern:
There has been a lot of concern from the OpenSolaris community about the silence from Oracle. Various community members believed that Oracle was just tying up Solaris 11. A new community based upon internal & external developers started the creation of Illumos in response to the silence. Illumos was discussed in the Network Management blog.


The Confirmation:
Jeff Burt at eWeek attended a live web event at Oracle on Tuesday August 06, 2010 with Oracle Executive Vice President of Systems, John Fowler. Jeff reported that Solaris 11 will be coming in 2011.

Oracle will release the next version of the Solaris operating system in 2011, and will double the performance of its SPARC processors every other year.

Sean Michael Kerner from ServerWatch also attended the live event, reporting the silence in the OpenSolaris community was due to the diligent work going on with the pending Solaris 11 release.

"Solaris 11 will be a superset of what is in openSolaris"... "We've been a little quiet on the open source front," Fowler said. "It's not that we're not investing in Solaris, we're just investing to make sure that we have all the major components for the new release."

The Odd Announcement:
Jeff Burt also reported in his eWeek article that Oracle Enterprise Linux is destined for SPARC?

Oracle will continue to support Oracle VM, its virtualization technology that enables businesses to run Windows and Linux environments—including Oracle's own Oracle Enterprise Linux—on SPARC-based systems.

The meaning of this phrase seems uncertain - one might be wise to wait for the real presentation material and transcript to be made available from Oracle.