Remember The Past, Forging The Future

Abstract:
There is value to remember history and listen to recent great thinkers when considering the future. The following are my highlights of the presentation by Bryan M. Cantrill of Joyent from a little over a half-year ago.

USENIX 2011 Presentation - Fork Yeah! The Rise and Development of illumos

• Introduction of History from SunOS, Solaris, to Illumos - 0:00-1:20
• SunOS 4.x , Solaris 2.0, Solaris 2.1, - 1:20-3:00
• Source Code Control: NSE, NSElite, Bitkeeper, TeamWare, to The Creation of GIT 3:00-4:00
• Solaris 2.3, Solaris 2.4, Solaris 2.5 (almost killed Solaris for BSD) - 4:00-5:20
• BDFL Software Model (Benevolent Dictator For Life) - 5:20-6:00
• The Bonwick Youth - 6:40-7:00; 8:41-10:00
• The Impact of Windows Scalability Day 1997 - 8:00-8:41
• Revolutionary Ideas 2001 - 10:00-12:00
• 1) ZFS - 12:00-12:15
• 2) DTrace - 12:15-13:00
• 3) Kevlar/Zones - 13:00-14:00
• 4-8) FMA, SMF, FireEngine, Crossbow, Least Privilege- 14:45-15:44
• Sun: Feuding Bands of Warlords - 15:44-16:00
• Innovation from Engineers, not Management, Marketing, Customer - 16:45-18:20
• Kiln of unspeakable pain - 17:30-18:00
• People, Not Organizations, Innovate - 18:00-18:20
• OS is a Loss Leader - 18:40-19:00
• Sun was The Open Systems Company (ex. NFS) - 19:40-20:00
• The Open Sourcing of Solaris - 20:00-21:00
• The Crap Code, not Innovative, costs money - 21:00-21:20
• The First Open Source, DTrace - 22:00-22:30
• Proprietary Drivers, CDDL Licensing - 23:15-25:00
• Fork-a-phobic, Boards, Elections, Governance, Politics - 27:00-28:00
• Copyright Assignment Concerns - 29:00-29:30
• OpenSolaris Missile Crisis, independent OGB - 30:00-33:00
• End of an Era, Oracle bought Sun - 33:00-34:00
  Kicked butt, had fun, didn't cheat, loved customers, changed computing forever
• Birth of Illumos - 39:50-41:20
• 2010 Friday 13th memo, Death of OpenSolaris - 41:20-43:50
• Solaris Diaspora - 44:00-45:30
• Illumos Innovation, No Illumos Copyright - 45:30-46:50
• Joint Working Group for ZFS - 47:00-50:00
  Feature flag, compression ratio, estimated zfs send/receive, Zone I/O throttling, ZFS unmap, background destroy, resumable send
• DTrace - 50:00-52:00
  Log-linear quantization, KVM support for vmregs, tracemem(), toupper(), tolower(), etc.
• Zones - 52:00-55:00
  More multi-tenancy under Illumos than Solaris, svcs -Z, svcs -L, per-zone kstat, new rcapd,
• KVM - 55:00-57:00
  Windows, Linux, BSD on near bare-metal speed (ZFS, Dtrace, Zones); KVM in a Zone
• Black Hat: "Break out of QEMU into a more secure cell" - 57:00-57:17
• Illumos Distributions - 57:30-58:00
• Illumos Community Values - 58:00-58:30
  Freedom to Fork; Benevolent Oligarchy; Value Utility
  "We reject: kings, presidents, voting.We believe in: rough concensus, running code" 

Foot Note:
History has consequences - not long after this, Solaris 11 source code was leaked from Oracle... of which, ironically, no one wants to use. Open Sourced Components bundled with Solaris 11 can be found here.

Articles of Interest: 06-04-2012

An interesting set of articles tracing progress in ZFS, Solaris, and other OpenSolaris based distributions.

• Larry Wake posted a note for people wishing to quickly find articles on the Oracle blog site about Oracle technologies. Articles tagged as Solaris, ZFS, IPS, DTrace, Zones, Studio, Cluster.


• Chris Mellor from The Register wrote an article about a new ZFS storage appliance Zebi by Tegile.


• The Art of Virtualization has a recent article regarding P2V (Physical to Virtual) Solaris migration.


• Henkis of the SPARCv9 blog discusses the Illumian distribution release, based upon Illumos.


• Wim Coekaert posted a short article titled "The case for Oracle VM", which discusses the pricing for Oracle VM without Oracle hardware support, as well as detailing how Oracle VM is free with Oracle Hardware Support.


• ZFS comes of age, with Open Source community adding features not bundled by Oracle. Async Destroy of filesystems is being implemented under Delphix with Feature Flags, to enable multiple ZFS development communities to add features without stepping upon one another.


• Async Destroy being integrated into Illumos by Delphix Chris Siden.


• Feature Flag being integrated into Illumos by Delphix Chris Siden.


• Chris Siden from Delphix discussed "ZTest" - a ZFS backwards-compatibility testing suite in early 2012.

Oracle Solaris 11: Session 3: Why We Moved to Oracle Solaris 11

Oracle Solaris 11: Session 3: Customers and Partners: Why We Moved to Oracle Solaris 11

Online Forum
Oracle Solaris 11:
What's New Since the Launch
April 2012

Abstract:
Oracle released a series of 4 sessions on Oracle Solaris 11: What's New Since the Launch (see Session 1 and Session 2.) Various customers described by they chose to implement under Solaris. Some of the more interesting customers made decisions to move from VMWare to Solaris Zones.

Viewbiquity: Moved From RAM to Solaris, ZFS, and SSD's
Viewbiquity ran their cloud on a system based upon adding RAM to their platform as data scaled.With the move to ZFS, 4.5x data compression noted using ZFS, performance increased by merely adding SSD's, splitting mirrors across chassis (and rotate mirror off-site.)

BH Photo: Moving to Solaris 11
Immutable zones, Least Privilege for users in Zones, higher performance, regulatory requirements for one-function-per-server using zones, move zones between different hardware type (T or M) to resolve performance problems.

 

 

Oppenheimer: Moving from Windows/VMWare to Zones/Solaris
Reduced downtime for patching, reduced security footprint for account requirement using role-based administration, zones are invisible as far as storage and system capacity, 20 zones on a server.

 

Wells Fargo: Consolidation by Business Function
Reduce maintenance windows, patching with high-availability requires less technical resources (without splitting mirrors), 40K server environment with 40% human error downtime is being eliminated, zones for tier-1 applications for higher-availability, able to move zones directly into Solaris 11 environment seamlessly.

ULM University: Patch, Availability, and Live Debugging with DTrace
Patch management is easy with ZFS taking snapshot (allowing for rollback.) Patching takes a minute with fast reboot. DTrace is one of the most "outstanding tools of my career", outstanding uptime, makes administrators more relaxed.

MSC Software: Solaris 11 and Certification of Applications in Partner Remote Lab
Scheduled to do performance testing in the summer of SPARC SuperCluster. Oracle Solaris Remote Lab, to certify their applications. Oracle Partner Lab provides zones for customers to get qualified easily under Solaris 11. Virtual machines using latest solaris, database, and middleware - completely pre-configured, taking minutes to turn up for partners.

SAS: Solaris OS Differentiates with ZFS, Zones, and DTrace
Long time partner, SPARC in 1993, x86 in 2006. 30 Million lines of portable code between Intel and SPARC. Excellent relationship with Oracle Solaris Studio for development. ZFS, Zones, and DTrace differentiate product from all other vendors on the market. High performance without tuning. Guaranteed binary compatibility is an advantage as a cost savings. The November 1996 release of SAS still works, when loaded under Solaris 11.

Solaris and Virtualization: KVM and Xen

[Solaris Logo, courtesy former Sun Microsystems]

Solaris and Virtualization: KVM and Xen

Abstract:
Solaris Intel looked to be a worthy operating system for the Intel marketspace with key technolgy such as ZFS, DTrace, Containers, and Xen. With Oracle's killing of Xen for Solaris 11 Intel, many were concerned that the benefits of Solaris on a server would be lost, without platform virtualization such as Xen. The Open Source Solaris community decided to invest in another technology, bringing true managed services scalability to the internet cloud environment.

[kvm logo, courtesy linux-kvm.org]

What is KVM:
KVM is the traditional acronym for "Keyboard, Video, Mouse" - as in KVM Switch. Unfortunately, some not-so-genius decided to overload the multi-decade old acronym with a new moniker - "Kernel Virtual Machine." The QEMU emulator is used as the foundation for the virtual machine. KVM was implemented to be tightly tied to the Linux kernel, with little attempt to be open and friendly to other system kernels, as was Xen.

[Joyent logo, courtesy joyent.com]

Cloud Computing: Joyent
Joyent moved KVM to the cloud with Solaris on their managed services grade SmartOS - encapsulating the finest of today's OS technologies: Zones, ZFS, DTrace, Crossbow. The use of KVM and QEMU within a Solaris Zone securely protects the hypervisor from virtual machine hacking exploits, which would ordinarily result in a disaster under Linux. The use of Crossbow provide for securiy and capping of network resources, through the underlying Solaris OS Kernel, to contain network exploits in the guest operating systems and/or their applicaions. Solaris DTrace provides unprecendeted KVM and guest operating system visibility - which includes cloud analytics from the hypervisor, operating system, middleware, and all the way down to the application layers.

Experiences porting KVM to SmartOS from bcantrill

The porting of  KVM to be encapsulated in a Solaris Zone within SmartOS was genius.

[Xen Logo, courtesy xenproject.org]

Conclusion:

The death of Xen under Solaris at the hands of Oracle was an untimely demise for a great concept - robust cloud computing could have been owned by Solaris, making others vendors look like a cheap knock-offs. While Oracle may have marketed Solaris 11 as the operating system of cloud computing, the real innovation came from Joyent, who implemented a multi-vendor cloud on top of managed services grade OpenSolaris fork, through the port of highly proprietary (operating system specific) KVM. Kudos to SmartOS team! Will HP, IBM, or Dell purchase Joyent for their SmartOS?

DTrace: Real Visibility

DTrace: Real Visibility

Abstract:
DTrace was developed by engineers from Sun Microsystems, and leveraged in their Fishworks project, to provide instrumentation for their Solaris 11 based storage system. After many of the original developers were cut from the staff, during the acquisition by Oracle, the DTrace community continued to significant diversify. Real visibility, to running applications and operating systems, is now reaching critical mass through the DTrace diaspora.


Oracle Announcement:
Oracle announced in Cotober 2011 that DTrace was being brought to Oracle Enterprise Linux. This announcement generated much discussion, but the history is fairly significant.

Some History:
There will be no attempt to provide an exhaustive history, but it is important to "travel down the roads least traveled" to gain an understanding of DTrace.

June 17, 2004 - Sun's Adam Leventhal starts work with Solaris DTrace
December 13, 2005 - Sun's Adam Leventhal discusses Solaris DTrace in a Linux Branded Zone
March 3, 2007 - Mailing list discussions regarding DTrace port to FreeBSD
August 2, 2007 - Sun's Adam Leventhal discusses knock-off DTrace copy on Linux
August 6, 2007 - Sun's Adam Leventhal discusses DTrace port to Linux
April 3, 2008 - Crisp developer Paul Fox starts DTrace port to Linux
June 30, 2008 - Sun's Adam Leventhal discusses a firestorm regarding DTrace port to Linux
August 10, 2010 - Oracle's Adam Leventhal departs and joins Delphix
September 23, 2010 - Oracle's Brendan Greg discusses upcoming DTrace Book
October 25, 2010 - Oracle's DTrace developer Brendan Greg departs for Joyent
November 19, 2010 - Oracle's Darren Moffat uses DTrace to better understand ZFS encryption
October 4, 2011 - Oracle announces DTrace for Oracle Enterprise Linux
October 5, 2011 - Delphix's Adam Leventhal discusses Oracle Linux DTrace alpha-beta
October 10, 2o11 - Delphix's Adam Leventhal discusses Oracle Linux DTrace Port issues
October 11, 2011 - Oracle's Wim Coekaerts "kicks the tires" on Oracle Linux DTrace
October 14, 2011 - Crisp developer Paul Fox discusses his DTrace Ubunto 11.10 port

Solaris Page Update: How To Use DTrace

Solaris Page Update: How To Use DTrace

The Network Management Solaris page has been updated, adding a reference to DTrace.

There is a feature embedded into Solaris, starting with Solaris 10, which allows for systems administrators to get telemetry (i.e. live data & statistics) from a running system with virtually no overhead. This is accomplished by the instrumentation of the OS (i.e. Solaris) at the kernel level, with something that is called "probe points". Any common systems administrator can diagnose a third party application performance by observing the Solaris "probe points" and measuring performance via DTrace.

Does it still sound strange?

It very well could, unless you have an operating system which is BSD derived. Apple OSX and Solaris are two operating systems which leverage this free infrastructure. Network Management resources should be skilled in this feature, to understand the scalability of their applications, operating systems, and hardware platforms.

Solaris Reference Material

2008/2010 [HTML] Solaris Dynamic Tracing Guide
2010-04 [PDF] Solaris 10/11 How to Use Oracle DTrace

Solaris DTrace

• DTrace.org - Blog from a creator of DTrace
• Solaris Dynamic Tracing Guide
• How To Use DTrace
• Introduction to the DTrace Book [Video]
• What is in the DTrace Book [Video]
• DTrace Book Released! (Book Samples: Table of Contents, File System Chapter)
• Observing File System Latency with MySQL using DTrace: Part [1|2|3|4]
• DTrace with Oracle VM Server for SPARC 2.1

Solaris LDoms / Oracle VM Server SPARC

• Oracle VM Server for SPARC Documentation
• DTrace with Oracle VM Server for SPARC 2.1

DTrace: Managing Applications in Modern Operating Systems

DTrace: Managing Applications in Modern Operating Systems

Abstract: Modern operating systems often have many commands at different layers in the software stack which are required in order to debug issues. DTrace changes this and provides a secure single interface to investigate nearly every layer in the software stack in a running production system, where developers can even create their own hooks for when they may need additional future insight.

Video interview on DTrace:
http://blogs.sun.com/video/entry/dtrace_for_system_administrators_with

PDF presentation on DTrace:
http://blogs.sun.com/brendan/resource/OOW2010_DTrace.pdf

The DTrace book:
http://www.amazon.com/dp/0132091518

Disk monitoring using DTrace:
http://www.princeton.edu/~unix/Solaris/troubleshoot/diskio.html
http://developers.sun.com/solaris/articles/dtrace_tutorial.html
http://wikis.sun.com/display/DTrace/io+Provider

Least Privilege with DTrace - No root required:
http://www.sun.com/bigadmin/features/articles/least_privilege.jsp

IBM: Where SUN Has Been Competitive

There have been rumors about IBM purchasing SUN for a number of days now.

Where has SUN been competitive, where IBM would want to purchase them?

Since 2007, SUN has dominated performance of SPECWeb 2006 in 1 socket - it takes 4 socket systems to edge out a 1 socket SUN T2 processor - and of course, you just buy a 2, 3, or 4 socket T2 system to crush competing results, by an order of magnitude.

For most of 2008, SUN was at the top of the list for CINT2006 Rates in 1 socket

Since 2008 and so far into 2009, SUN is in the top 3 lists for in CINT2006 Rates in 2 sockets
http://spec.org/cgi-bin/osgresults?conf=rint2006&op=fetch&proj-COMPANY=256&proj-SYSTEM=256&proj-CORES=256&proj-CHIPS=256&critop-CHIPS=0&crit-CHIPS=2&proj-CORESCHP=256&proj-THREADS=0&proj-CPU=0&proj-CPU_MHZ=0&proj-CPUCHAR=0&proj-NCPUORD=0&proj-PARALLEL=0&proj-BASEPTR=0&proj-PEAKPTR=0&proj-CACHE1=0&proj-CACHE2=0&proj-CACHE3=0&proj-OCACHE=0&proj-MEMORY=0&proj-OS=0&proj-FS=0&proj-COMPILER=0&proj-HWAVAIL=0&crit2-HWAVAIL=Jan&proj-SWAVAIL=0&crit2-SWAVAIL=Jan&proj-COPIES=256&proj-PEAK=256&proj-BASE=256&proj-400PEAK=0&proj-400BASE=0&proj-401PEAK=0&proj-401BASE=0&proj-403PEAK=0&proj-403BASE=0&proj-429PEAK=0&proj-429BASE=0&proj-445PEAK=0&proj-445BASE=0&proj-456PEAK=0&proj-456BASE=0&proj-458PEAK=0&proj-458BASE=0&proj-462PEAK=0&proj-462BASE=0&proj-464PEAK=0&proj-464BASE=0&proj-471PEAK=0&proj-471BASE=0&proj-473PEAK=0&proj-473BASE=0&proj-483PEAK=0&proj-483BASE=0&proj-LICENSE=0&proj-TESTER=0&proj-SPONSOR=0&proj-TESTDAT=0&crit2-TESTDAT=Jan&proj-PUBLISH=256&critop-PUBLISH=-1&crit2-PUBLISH=Mar&crit-PUBLISH=2009&proj-UPDATE=0&crit2-UPDATE=Jan&dups=0&duplist=COMPANY&duplist=SYSTEM&duplist=CORES&duplist=CHIPS&duplist=CORESCHP&duplist=THREADS&duplist=CPU&duplist=PARALLEL&duplist=BASEPTR&duplist=PEAKPTR&duplist=CACHE1&duplist=CACHE2&duplist=CACHE3&duplist=OCACHE&duplist=COPIES&dupkey=PUBLISH&latest=Dec-9999&sort1=PEAK&sdir1=-1&sort2=SYSTEM&sdir2=1&sort3=CORESCHP&sdir3=1&format=tab

Since 2008 and so far into 2009, SUN is the top CINT2006 Rates in 4 sockets, even pulling away from the quad hex-core Intel processors by 25%

For most of 2008, SUN was at the top of the list for CFP2006 Rates in 1 socket.

For all of 2008 and so far though 2009, SUN is at the top of the list for CFP2006 Rates in 2 socket.

Since end of 2008 and so far though 2009, SUN is at the top of the list for CFP2006 Rates in 4 sockets.

SPARC has been clearly very competitive FOR YEARS on a field where POWER had decided to not even compete.

SUN has been very competitive in the Applications Arena. OpenOffice is a SUN led OpenSource project, which IBM rebranded as Lotus Symphony, and SUN rebrands as StarOffice.

SUN has been very competitive in the Application Foundation arena. Most cross-platform enterprise applications are written in JAVA.

SUN has been very competitive in Tape Storage. SUN and IBM are basically the only games in town for substantial tape storage. The American Government would probably demand a spin-off of something in the case of a merger.

SUN has been very competitive in the OS arena. There is nothing in the market like Open Source Solaris, only Windows has a more complete CIFS Kernel API set than OpenSolaris for CIFS/SMB. Only OpenSolaris offers a very full featured file system like ZFS (other OS's like Apple are starting to port and adopt pieces into MacOSX.) Systems administrators being able to trace third party applications programaticaly using DTrace is unheard of in the industry.

SUN has been very competitive in the Ultra-Thin Client arena. With third party manufacturers making ultra-thin laptops, SUN making ultra-thin desktops, significant power consumption savings from these units (better than any competing platform), and demonstrated savings over thens of thousand of users (SUN's policy of Eat Your Own Dogfood as well as U.S. Department of Defense) places SUN in a very unique position to share Solaris, Windows, and Linux applications securely over WiFi, Ethernet, and Fiber with zero desktop management (unless you consider unplugging and throwing out your box and moving to a different station to pick up from where you left off a problem!)

Considering the size of SUN and the resources of the competition, SUN has done fairly well, even if they have not been able to compete everywhere as effectively as those companies with deeper pockets such as IBM.