MacOSX: Zevo ZFS Moved to GreenBytes

Ten's Complement, ZEVO, ZFS, and GreenBytes
Nearly 6 months ago, Network Management discussed the history of moving technological market leading file system ZFS to MacOSX by Ten's Complement. The product was named ZEVO. Storage vendor GreenBytes purchased ZEVO technology, suggesting it will continue to be developed.

The ZEVO ZFS technology for Mac, developed by former Apple engineer Don Brady at his company Ten's Complement, has been acquired by enterprise storage vendor GreenBytes. Brady announced the transition on Twitter and on the Ten's Complement site; he is joining the GreenBytes development team.

Network Management
It is appearing like there may be a new Storage Vendor, thinking about moving to ZFS, and possibly binding itself a little closer to Apple systems.

Midrange CPU Board Basics (Part 1)

Abstract:

Every so often, when working on a Sun server, it is helpful to know the positioning and speed of the CPU boards, to plan for better upgrades. This article takes a few common machines and provides some basic, simple to read instructions, for determining CPU capabilities.

Sun V490 (SUNW,Sun-Fire-V490)

Introduction

The Sun Microsystems server Sun Fire V490 was a machine on the high end of the workgroup servers. This server has a 2 CPU board capacity, where each CPU board holds 2 sockets, where each socket typically holds 2 cores.

These are not Intel cores, but each core addition to the socket increase performance close to linearly (instead of by 50% in Intel or AMD sockets of this age.)

Determining Class

The "uname" provides for an easy way to know the class of machine.

sun1316$ uname -a
SunOS sun1316 5.9 Generic_122300-57 sun4u sparc SUNW,Sun-Fire-V490

The name of the platform is a "Sun-Fire-V490", hinting this chassis is capable of 4 sockets. The "90" indicates it was an UltraSPARC IV based machine, which was capable of dual-cores. (The chassis is also compatible with older single board UltraSPARC III processors.)

Determining Boards

The psrinfo command is available in the /usr/sbin directory.

sun1316$ /usr/sbin/psrinfo
0       on-line   since 07/15/2012 02:31:01
1       on-line   since 07/15/2012 02:31:01
2       on-line   since 07/15/2012 02:31:01
3       on-line   since 07/15/2012 02:31:00
16      on-line   since 07/15/2012 02:31:01
17      on-line   since 07/15/2012 02:31:01
18      on-line   since 07/15/2012 02:31:01
19      on-line   since 07/15/2012 02:31:01

Processors 16 and over are an indication that this socket is a dual-core socket. On this chassis, sockets 0-1 are located on board 1, while sockets 2-3 are located on board 1. Both boards are populated.

Determining Performance

The psrinfo with a "-v" option will provide additional information, such as the speed of the individual cores.

sun1316$ psrinfo -v
Status of virtual processor 0 as of: 07/19/2012 22:43:35
  on-line since 07/15/2012 02:31:01.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 07/19/2012 22:43:35
  on-line since 07/15/2012 02:31:01.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 07/19/2012 22:43:35
  on-line since 07/15/2012 02:31:01.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 07/19/2012 22:43:35
  on-line since 07/15/2012 02:31:00.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
...

In the above example, I trimmed the output of the second cores on the 4 sockets, since the information is identical to the first 4 cores on each socket.

Sun V890 (Sun-Fire-V890)

Introduction

The Sun V890 was a machine square in the high end of the workgroup servers. The server has a 4 CPU board capacity, where each CPU board holds 2 sockets, where each socket typically holds 2 cores.

These are not Intel cores, but each core addition to the socket increase performance close to linearly (instead of by 50% in Intel or AMD sockets of this age.)

Determining Class

sun1375$ uname -a
SunOS sun1376 5.10 Generic_144488-11 sun4u sparc SUNW,Sun-Fire-V890

The name of the platform is "V890" hinting at 8 socket capability. The "90" hints it was capable of using dual-core UltraSPARC processors. The chassis was capable of using UltraSPARC III boards.

Determining Boards

The psrinfo command is available in the /usr/sbin directory.

sun1376$ psrinfo 
0       on-line   since 05/16/2011 14:26:00
1       on-line   since 05/16/2011 14:26:00
2       on-line   since 05/16/2011 14:26:00
3       on-line   since 05/16/2011 14:26:00
4       on-line   since 05/16/2011 14:26:00
5       on-line   since 05/16/2011 14:26:00
6       on-line   since 05/16/2011 14:26:00
7       on-line   since 05/16/2011 14:25:55
16      on-line   since 05/16/2011 14:26:00
17      on-line   since 05/16/2011 14:26:00
18      on-line   since 05/16/2011 14:26:00
19      on-line   since 05/16/2011 14:26:00
20      on-line   since 05/16/2011 14:26:00
21      on-line   since 05/16/2011 14:26:00
22      on-line   since 05/16/2011 14:26:00
23      on-line   since 05/16/2011 14:26:00

Processors 16 and over are an indication that this socket is a dual-core socket. On this chassis, sockets 0-1 are located on board 1, sockets 2-3 are located on board 2, sockets 4-5 on board 3, sockets 6-7 on board 4. All boards are populated.

Determining Performance

The psrinfo with a "-v" option will provide additional information, such as the speed of the individual cores.

sun1376$ psrinfo -v
Status of virtual processor 0 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:26:00.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:26:00.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:26:00.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:26:00.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 4 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:26:00.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 5 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:26:00.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 6 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:26:00.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 7 as of: 07/19/2012 22:55:41
  on-line since 05/16/2011 14:25:55.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
...

In the above example, I trimmed the output of the second cores on the 4 sockets, since the information is identical to the first core on each socket. Note, the cores ran at 1.5GHz.

Sun E2900 (SUNW,Netra-T12)

Introduction

The Sun E2900 was a machine on the cusp of between a workgroup and a midrange server. This server has a 3 CPU boards capacity, where each CPU board holds 4 sockets, where each socket typically holds 2 cores.

These are not Intel cores, but each core addition to the socket increase performance close to linearly (instead of by 50% in Intel or AMD sockets of this age.)

Determining Class

The "uname" provides for an easy way to know the class of machine.

sun1142$ uname -a
SunOS sun1142 5.10 Generic_138888-07 sun4u sparc SUNW,Netra-T12

The name of the platform is a "SUNW-Netra-T12", hinting this chassis is capable of 12 sockets. The "900" in the 2900 model is a hint indicating this chassis is capable of using UltraSPARC IV dual-core CPU's. (The chassis is also compatible with older single board UltraSPARC III processors.)

Determining Boards

The psrinfo command is available in the /usr/sbin directory.

sun1142$ /usr/sbin/psrinfo
0       on-line   since 02/12/2012 02:27:26
1       on-line   since 02/12/2012 02:27:42
2       on-line   since 02/12/2012 02:27:42
3       on-line   since 02/12/2012 02:27:42
8       on-line   since 02/12/2012 02:27:42
9       on-line   since 02/12/2012 02:27:42
10      on-line   since 02/12/2012 02:27:42
11      on-line   since 02/12/2012 02:27:42
512     on-line   since 02/12/2012 02:27:42
513     on-line   since 02/12/2012 02:27:42
514     on-line   since 02/12/2012 02:27:42
515     on-line   since 02/12/2012 02:27:42
520     on-line   since 02/12/2012 02:27:42
521     on-line   since 02/12/2012 02:27:42
522     on-line   since 02/12/2012 02:27:42
523     on-line   since 02/12/2012 02:27:42

Processor 512 and over are an indication that this socket is a dual-core socket. On this chassis, sockets 0-4 are located on board 1, while sockets 8-11 are located on board 3. Board 2 is missing.

Determining Performance

The psrinfo with a "-v" option will provide additional information, such as the speed of the individual cores.

sun1142$ /usr/sbin/psrinfo -v
Status of virtual processor 0 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:26.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:42.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:42.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:42.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 8 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:42.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 9 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:42.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 10 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:42.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 11 as of: 07/19/2012 22:20:39
  on-line since 02/12/2012 02:27:42.
  The sparcv9 processor operates at 1950 MHz,
        and has a sparcv9 floating point processor.
...

Note, on this chassis, a board runs at a uniform clock rate across all sockets and cores, so only one core per board is needed, but I ignored the second cores in each socket to shorten the above example. Board 1 and Board 3 both use 1.95GHz clock rate. 2.1 GHz is the fastest board which can be purchased for this chassis.

An example of a completely filled chassis with differing speed CPU boards is as follows:

sun1143$ /usr/sbin/psrinfo -v
Status of virtual processor 0 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:08.
  The sparcv9 processor operates at 1200 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1200 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1200 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1200 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 8 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 9 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 10 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 11 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1350 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 16 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 17 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 18 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 19 as of: 07/19/2012 22:32:18
  on-line since 07/17/2012 11:27:26.
  The sparcv9 processor operates at 1500 MHz,
        and has a sparcv9 floating point processor.
...

Note, in the above example, I cut out the second cores, to simplify the output. It can be seen that the 3 CPU boards are running at 1.5GHz, 1.35GHz, and and 1.2GHz

Network Management
Most network management platforms require excessive uptime and outstanding expandability. One of the reasons for choosing platforms such as these would be to have robust platforms which will survive for the duration of a contract with an end customer. This may be 3-5 years.

Many of these mid-range platforms are coming towards end of managed services contract, and there is a lot of horse power left in them, to provide services for the next 3-5 year range. Adding a new CPU and Memory card can extend the capital investment of the asset for years to come, if you know which platforms can be extended, and know which platforms to retire.

Often, older high-end platforms are rotated to become development platforms, while older low-end platforms are retired. One of the difficulties experienced by network management providers center around the ISV's, who have had their legs chopped out from under them, by Oracle not releasing Solaris 11 under these UltraSPARC units. Some ISV's have just chosen to stop developing for SPARC Solaris because the barrier to entry is now too high (must buy new development and new production SPARC hardware... not to mention the lack of inexpensive Solaris SPARC desktops.)

Why bother developing network management applications for Solaris SPARC? The next generation of SPARC processors are terrific: crypto engines, 128 slow threads, 64 fast threads, binary compatibility for nearly forever, future 128 fast threads, future 8 socket platforms, built in gigabit ethernet in the CPU socket, etc.

It is good to know that the SPARC also has a second supplier, which bailed out Sun Microsystems a number of years ago, when they were not doing so well with their advanced processor lines. It is comforting to know that viruses and worms seldom target SPARC. It is also good to know that a platform which is booted yearly provides availability unlike most others - and Network Management is all about Availability.

From Cloud 1.0, to Cloud 2.0, to Cloud 3.0

Abstract:
Meg Bear, Vice President, Oracle Cloud Social Platform, published an opinion blog article titled "Multi-Tenancy and Other Useless Discussions." Cited were two articles regarding "very strong opinions on either side of the multi-tenancy divide." The irony in this opinion article is that Meg does not seem to understand that Oracle has the technological lead among the giants in this arena, but are quickly losing their position. A subset of this post was submitted in a comment on Wednesday July 18th, a little after mid-day.

Oracle's Position
With the advance of massive ZFS storage capabilities from Oracle and hypervisors - there is tremendous opportunity for cloud based solutions to provide customers with incredible customization options.

Multi-tenancy offers the ability to offer consistent levels of service and features across multiple customers. Multi-tenancy also offers reduced overhead for the service provider to reduce costs in managing the solutions - which leads to greater application availability. Oracle has superior products which technologically corner this market.

A multi-tenant solution does not mean that every customer has to be on the same version of the software. New versions of the software can be rolled out in parallel and customers can "choose" which version they wish to be under... the former release or the newer release. Good multi-tenant solutions should never be monolythic, rather they should be modular and parallel.

The less capable an individual software solution is, the less isolation, scalability, and management features are offer. Less mature solution do not offer multi-tenancy. It is not necessarily an issue with "Cloud V1" vs "Cloud V2", as the Oracle CEO suggested, it is an issue with cloud solution maturity.



Let us illustrate:
The Oracle VM for SPARC hypervisor offers advanced Oracle ZFS filesystem capabilities with: massive dataset capability; encrypted datasets on a per-customer basis for storage, over-the-wire, and memory buffers for security; superior performance with hardware acceleration of compression (T5) and encryption (T1-T5); guaranteed data integrity & correction from the OS, through hypervisor, through memory, through HBA, over wire, to storage, and to the disk (at every layer of the stack); and much higher capacity & throughput via ZFS deduplication in memory, via hypervisor, over-the-wire, and on disk storage (and not just over the disk bus); and vastly superior visibility and analytics for live production systems with no application interruption with DTrace at every level of the hypervisor, to OS Kernel, through Java, to the application. Oracle owns all of this multi-tenant technology under "Cloud 1.0" - nearly everyone else just borrows some pieces to produce a popular (but technologically inferior) solution.


Why is this brought up?
These capabilities are notably missing from the Oracle VM for Intel hypervisor - the basis for "Cloud 2.0". Clouds based upon this solution are not as robust. Moving the ZFS to a dedicated storage infrastructure leaves gaping holes. From a cloud service delivery perspective, ZFS would need to run on the hypervisor or OS layer in order to: correct hidden data corruption introduced at and below the hypervisor layer (memory, backplane, HBA, wiring levels, etc.); provide throughput improvement via compression & deduplication (via memory, backplane, hypervisor, over-the-wire, storage subsystem); and provide massive storage capacity.


Security, data integrity, and performance drive multi-tenancy requirements. Applications run under "Cloud 1.0" and "Cloud 2.0" solutions. Oracle VM for Intel hypervisor needs to "get with the program" and gain some of these [nearly 10 year old] security, data integrity, and performance technologies leveraged by service providers in the traditional "Cloud 1.0" multi-tenant market as well as by some "Cloud 2.0" (Oracle-derived, but now independent) service providers.

Right now, there seems to be 3 technologies which completely satisfy market needs suggested by the CEO in "Cloud 2.0": Oracle VM for SPARC clouds (Oracle leaving this for other Multi-Tenant providers); Xen for Solaris Intel clouds (Oracle discarded); and KVM for OpenSolaris under Intel (Joyent produced SmartOS, based upon Illumos, based upon OpenSolaris.)


ZFS, Encryption, Compression, Deduplication, and Visibility are the technologies today. Filesystem clustering is tomorrow. Everything else is "just a cloud" floating by... based upon commodity Linux, without data integrity, without data security, without performance, without visibility, without clustering - but there is a thunderhead on the horizon.

Clustered filesystems in conjunction with ZFS fills a bigger hole, in what should be called the "Cloud 3.0" market. Honestly, a Cloud should use clustered filesystems with the data everywhere, with the data encrypted everywhere, with the data compressed everywhere, with the data deduplicated everywhere (and, of course, this means memory, the wire, and the storage.)


Intel x64, IBM POWER, and Oracle [any-architecture]
The HPC market is nothing more than a specialized cloud. Intel purchasing Whamcloud, for Oracle Lustre on Linus's Linux on Intel's x64, and IBM, layering Oracle Lustre on Oracle ZFS on Linus's Linux on IBM POWER, shows where the cloud market is moving towards. IBM pulling off what Oracle could not do, years after the purchase of Sun Microsystems. The market is clearly and proverbially "throwing down the gauntlet".

Oracle is the only vendor on the horizon who has the possibility of clearly layering Clustering on ZFS with full visibility, without sacrificing the performance of a Kernel implementation of ZFS and Lustre, under Oracle Solaris with a Solaris hypervisor, to produce (what I would refer to as) "Cloud 3.0".


If Oracle does not capitalize on "Cloud 3.0" by bundling Lustre, ZFS, and Solaris (on Intel with Xen, SPARC with LDom's) - Oracle "Cloud 1.0" multi-tenancy will still continue to be the 1000 pound gorilla in the room. Oracle (and nearly every other vendor, outside of Joyent) are "Cloud 2.0" Resus Monkies, lacking DTrace visibility and in-kernel clustering & ZFS. "Cloud 3.0" has the potential to dominate Oracle in technical capability, using derived Oracle technologies like ZFS, Lustre, and Xen (which both Oracle and Sun both participated in.)

With every technology Oracle ports to Oracle Linux, the more technology is "given away" to it's competitors, as it is absorbed "upstream" and sucked into Red Hat and Suse - who are recognized Linux vendors.

Network Management:
Network Management platforms used by Telecommunication Service Providers are dependent upon large multi-tenant software solutions. With Oracle Linux not showing up on Network Management software vendor list, the only serious options are SPARC Solaris, Intel Red Hat Linux, and Intel Windows.

The lack of an integrated clustered filesystem with ZFS under Solaris, has long dogged the telco providers. The drive towards Red Hat was relentless. The push by Oracle to "Cloud 2.0", without ever providing a technologically superior "Cloud 2.0" alternative under Intel (Windows, Linux, or Solaris) has left Oracle largely out of the picture.

The drive to "Cloud 3.0" appears strangely dim for Oracle, while it looks promising for OpenSolaris dependent vendors like Joyent. OpenSolaris based technologies (Solaris 11, SmartOS, Illumian, OpenIndiana, etc.) are superior to the dominate market players, but this could be pre-empted by Intel or IBM, as they fund the contribution of Oracle technologies to Linus and his Linux - to suck all value out of Oracle's "Cloud V1" and provide superior "Cloud V2" solution to anything that Oracle is supplying today.

 

Remember The Past, Forging The Future

Abstract:
There is value to remember history and listen to recent great thinkers when considering the future. The following are my highlights of the presentation by Bryan M. Cantrill of Joyent from a little over a half-year ago.

USENIX 2011 Presentation - Fork Yeah! The Rise and Development of illumos

• Introduction of History from SunOS, Solaris, to Illumos - 0:00-1:20
• SunOS 4.x , Solaris 2.0, Solaris 2.1, - 1:20-3:00
• Source Code Control: NSE, NSElite, Bitkeeper, TeamWare, to The Creation of GIT 3:00-4:00
• Solaris 2.3, Solaris 2.4, Solaris 2.5 (almost killed Solaris for BSD) - 4:00-5:20
• BDFL Software Model (Benevolent Dictator For Life) - 5:20-6:00
• The Bonwick Youth - 6:40-7:00; 8:41-10:00
• The Impact of Windows Scalability Day 1997 - 8:00-8:41
• Revolutionary Ideas 2001 - 10:00-12:00
• 1) ZFS - 12:00-12:15
• 2) DTrace - 12:15-13:00
• 3) Kevlar/Zones - 13:00-14:00
• 4-8) FMA, SMF, FireEngine, Crossbow, Least Privilege- 14:45-15:44
• Sun: Feuding Bands of Warlords - 15:44-16:00
• Innovation from Engineers, not Management, Marketing, Customer - 16:45-18:20
• Kiln of unspeakable pain - 17:30-18:00
• People, Not Organizations, Innovate - 18:00-18:20
• OS is a Loss Leader - 18:40-19:00
• Sun was The Open Systems Company (ex. NFS) - 19:40-20:00
• The Open Sourcing of Solaris - 20:00-21:00
• The Crap Code, not Innovative, costs money - 21:00-21:20
• The First Open Source, DTrace - 22:00-22:30
• Proprietary Drivers, CDDL Licensing - 23:15-25:00
• Fork-a-phobic, Boards, Elections, Governance, Politics - 27:00-28:00
• Copyright Assignment Concerns - 29:00-29:30
• OpenSolaris Missile Crisis, independent OGB - 30:00-33:00
• End of an Era, Oracle bought Sun - 33:00-34:00
  Kicked butt, had fun, didn't cheat, loved customers, changed computing forever
• Birth of Illumos - 39:50-41:20
• 2010 Friday 13th memo, Death of OpenSolaris - 41:20-43:50
• Solaris Diaspora - 44:00-45:30
• Illumos Innovation, No Illumos Copyright - 45:30-46:50
• Joint Working Group for ZFS - 47:00-50:00
  Feature flag, compression ratio, estimated zfs send/receive, Zone I/O throttling, ZFS unmap, background destroy, resumable send
• DTrace - 50:00-52:00
  Log-linear quantization, KVM support for vmregs, tracemem(), toupper(), tolower(), etc.
• Zones - 52:00-55:00
  More multi-tenancy under Illumos than Solaris, svcs -Z, svcs -L, per-zone kstat, new rcapd,
• KVM - 55:00-57:00
  Windows, Linux, BSD on near bare-metal speed (ZFS, Dtrace, Zones); KVM in a Zone
• Black Hat: "Break out of QEMU into a more secure cell" - 57:00-57:17
• Illumos Distributions - 57:30-58:00
• Illumos Community Values - 58:00-58:30
  Freedom to Fork; Benevolent Oligarchy; Value Utility
  "We reject: kings, presidents, voting.We believe in: rough concensus, running code" 

Foot Note:
History has consequences - not long after this, Solaris 11 source code was leaked from Oracle... of which, ironically, no one wants to use. Open Sourced Components bundled with Solaris 11 can be found here.

Architecture Update: The ARMs Race

Abstract:
With the explosion of ARM processors in embedded systems, ranging from phones to tablets, ARM designers are creating ever more complex processors, CPU manufacturers are creating more options for system designers, and even system designers are discussing the movement of ARM from embedded to server and desktop systems.

CPU Architecture

64 Bit ARM v8
First, let's discuss the CPU architecture updates to the ARM processing architecture. Traditionally, the ARM processor bundled a 32 bit processor. While this was more than adequate for embedded systems, this limited the application of the architecture into other spheres of computing.



At the ARM TechCon conference in Santa Clara, California, in October 2011 - the ARM v8 64 bit architecture was announced, demonstrating 64 bit extensions similar to what was done with SPARC, AMD, and later Intel CPU's.

The new ARMv8 architecture has two execution states, the AArch32 state that is compatible with prior generations of 32-bit ARM processors, and AArch64, the new 64-bit extensions. At the moment, the ARMv8 architecture has only been profiled for what ARM calls the A line of its Cortex reference designs, which means they are designated for application processing such as that done on smartphones and tablets.

The ARMv8 architecture will bring forward TrustZone virtualization (which debuted with the ARM v6) and NEON SIMD instructions, which debuted with the ARM v7 designs. The interesting thing about the ARMv8 is that it will offer both double-precision floating point math through that NEON unit.

ARM Update: Mali 450 GPU moves from 4 to 8 cores

The Registered published a short article about an ARM roadmap split:

ARM is doubling the punch of its Mali 400 graphics processors with extra cores for tablet, phone and TV makers that are not ready for combined graphics and compute chips.
The microprocessor architect has announced the Mali 450 GPU, featuring eight cores instead of four.

ARM said the 450 showed it remains committed to the 400 range, and said it is now splitting its roadmap.

ARM Designs Quad-Core 32-bit v7
In April of 2011, ARM announced their quad-core Cortex-A15 processor was scheduled to appear in smart phones or tablets in 2012-2013.

ARM's Cortex-A15, however, will up the ante with an out-of-order superscalar pipeline, 40-bit memory-addressing capabilities, floating-point and media-handling improvements, and a clock speed of up to 2.5GHz, all at power requirements said to be comparable to the company's current Cortex-A9 design.

The Cortex-A9 is the design upon which such top-end smartphone and tablet chips as Apple's A5, Nvidia's Tegra 2, and Samsung's Exynos 4210 are based. The Cortex-A15 design, meanwhile, has already been licensed by Texas Instruments and Nvidia, and Nvidia



(ARM v8 Exception Model, courtesy ARMv8 Architecture PDF)

ARM Starts Designing 64-bit
Richard Grisenthwaite, Lead Architect at ARM, provided a technology preview of the ARMv8 64 bit architecture at ARM TechCon in 2011. It was clear from the document that 32 bit ARM processors would continue to be designed and that this was merely a new line of processor design which manufacturers could leverage.

System Designers

(A Boston Viridis server, front view, no cover, courtesy, The Register)



(Boston Limited's first Viridis server, courtsy The Register)



Boston

The Register writes about U.K. IT Supplier Boston is releasing their Viridis platform, based upon their Calexeda partnership, using the Smoothstone ARM processor.

"The Viridis server is using the 1.4GHz variant of the ECX-1000 processors and plunks a 4GB DDR3 memory stick in for each node on the card. The card has two 10GE network ports and four SATA disk ports per processor... The dozen processor cards including memory burn only 300 watts."





(Dell Quad ARM Server Chassis)



(Dell Quad ARM Server Blade)



Dell

It was mentioned during the June 2012 Network Management "System Vendor: CISC, RISC, EPIC Update"  that Dell was breaking into the ARM marketplace. Each Dell blade holds 4 32-bit ARM servers. Ironically, Dell's blade server looks a lot like a far less rugged old Sun Fire B1600 blade chassis, which contained 3U high SPARC RISC SPARC blades, but each Sun blade, from a decade ago, only held single 64 bit server.


(HP Redstone ARM v7 32 bit Server, courtesy The Register)

 HP Enters the ARMs Race
In November 2011, HP released ARM RISC servers to supplement their Proliant CISC servers and Itanium LWIS processors. This was the result of "Project Moonshot".

"To make the Redstone, HP took a half-width, single-height ProLiant tray server and ripped out just about everything but the tray. In goes the passive backplane that the Calxeda EnergyCard, and HP can cram three rows of these ARM boards, with six per row, for a total of 72 server nodes, in a half-width 2U slot... That gives you 288 server nodes in a 4U rack space, or 72 servers per rack unit."

ARM CPU Manufacturers

Samsung
DRAM manufacturer, and more recently cell phone manufacturer, appears to be hiring CPU designers, possibly for the ARM CPU chips, used in Apple and their own cell phones. Various chip designers are being harvested with experience ranging from Sun Microsystems and Oracle to AMD.

Calxeda
November 2011 - Calxeda announced their 32 bit quad-core EnergyCore ARM v7 EnergyCore ECX-1000 Series CPU's.

"Calxeda has spent the past several years tweaking the 32-bit ARMv7 core to come up with its own system-on-chip (SoC) and related interconnect fabric suitable for hyperscale parallel and distributed computing where nodes have only modest memory needs."

[Applied Micro CEP Paramesh Gopi, courtesty The Register]

Applied Micro
Also in October of 2011, Applied Micro announced their X-Gene 64 bit ARM v8 processors.

(X-Gene ARMv8 CPU)

"The X-Gene chip will also include DDR3 main memory controllers, two 10 Gigabit Ethernet ports, SATA storage and PCI-Express peripheral controllers, and a power/management module – all on the same die as the cores."
"The cores will have L1 and L2 caches per core, a shared L3 cache that spans the cores, and have a target clock speed of 3GHz."
"The X-Gene chip also has on-chip CPU and I/O virtualization, just like x86, Sparc, Power, and Itanium chips do. The architecture also allows for various kinds of offload engines to be plugged in and perhaps integrated on the chip package."

(X-Gene ARM v8 Block Diagram, courtesy The Register)

The X-Gene is suposed to be ready to ship second half of 2012 - which is right about now. Taiwan Semiconductor Manufacturing Corporation (TSMC) is first etching the chips using a 40nm process, with subsequent designes in 28nm.

Nvidia
At the Las Vegas Consumer Electronics Show in April 2011, video chip processing giant Nvidea discussed phones based upon their Tegra 2 dual-core ARM Cortex-A9 chips, which bundle graphics processing, licensed the future Cortex-A15 design, and announced "Project Denver" circa 2013 - targeting desktops.

"Denver provides a choice. System builders can now choose a high-performance processor based on a RISC instruction set with modern features such as fixed-width instructions, predication, and a large general register file. These features enable advanced compiler techniques and simplify implementation, ultimately leading to higher performance and a more energy-efficient processor."

Back in September of 2010, Nvidea president and CEP Jen-Hsun Huang also discussed their "Kepler" ARM processor, due in 2011, and the "Maxwell" ARM processor due in 2013.

(Armada XP Processor, courtesy The Register)

Marvell

Chip manufacturer Marvell acquired the ARM RISC CPU business from Xscale in 2006. In 2010, Marvell announced it's quad-core 32 bit ARM v7 Armada XP processor, implemented on a 40nm process.

"...running at 1.6 GHz with a shared 2 MB L2 cache memory... The chip will include variants that support 64-bit DDR2, DDR3, and DDR3 low-voltage memory chips. For on-chip DDR3 controllers, the memory can run at to 800 MHz and ... has ECC memory scrubbing."

"The chip includes four PCI-Express 2.0 x4 interfaces and four Gigabit Ethernet controllers etched into its silicon; it has 16 SERDES lanes for implementing USB, PCI-Express, SATA, SGMII, and QSGMII ports..."

 It seems 2013 could prove very interesting from Marvell.

Conclusions
It is very odd, not to see IBM producing any platforms based upon ARM, but very interesting to see IBM assisting ARM to reduce it's chip process down to 14nm, back in January of 2011. One has to wonder, at what point will IBM stop developing POWER (POWER 7+ is now about 10 months late?) or stop helping ARM produce smaller & faster processors. Up until this point, POWER was not in competition with ARM, but clearly ARM is climbing the food chain, moving to thin client desktops, cell phones, tablets, and now servers.

Apple Mac OSX, based upon BSD UNIX, and Google's Android Linux are the main OS players in the ARM arena - with Microsoft starting to produce Windows ports.

OpenSolaris port to ARM was of interest back in 2009, a code contribution made in 2009, additional work in Feb/Mar 2012 timeframe with some code, Illumos developers considering ARM in March 2012, Google "Summer of Code" ARM project idea suggested in April 2012, a grad student showing interest in April 2012, and with all the activity around ARM servers - one might hope that there will be additoinal interest in the Illumos community.

Will other OS vendors port to ARM?

Thunderbird: On The Back Burner

Abstract:
Mail has long been the backbone of large organizations. Mail was considered such a critical function, it was enshrined as a responsibility of the Federal Government in the founding of the United States. Businesses had secretaries who took memos. Businesses built internal mail infrastructure. Speed became more critical, noting The Pony Express. Written communications became virtual, with the the telegraph. EMail became one of the first functions of networking on The Internet, with all other communications being called "snail mail". EMail was built into UNIX desktop workstations. EMail was unintuitively debundled with a separate fee by a dominate desktop vendor. EMail became bundled into internet HTML access. A new change is happening in the Open community, which the computing community should be made aware of, regarding Mozilla's Thunderbird, the off-shoot of the original integrated email & web browser, Netscape.

History:
Early, UNIX was the core of Internet, with text based email clients running with internet standard SMTP (Simple Mail Transfer Protocol.) UNIX workstations, such as the original Sun Microsystems, started bulding multi-media graphical email clients as part of their operating system (SunOS) and desktop environment (SunView), free of charge. With the investment from AT&T, Sun migrated to SVR4 base, and cooperated with AT&T on OpenLook - migrating their desktop application suite (including email) to make OpenWindows. Other vendors cooperated to build Motif. Open Systems vendors combined their efforts together to merge Motif with the Sun Desktop clients to become CDE (Common Desktop Environment) - which bundled the Sun desktop widget set, including the graphical email client.

AT&T did not offer a multimedia email client on their PC based Intel SVR4 OpenLook environment, but you could buy a Microsoft Mail server for AT&T UNIX SVR4. AT&T strangely decided to  "give up" the email market, which it was partially responsible in building through UNIX. Their mail would remain command line driven. Fact is stranger than fiction, sometimes - they will wind up funding other people's development, in the future.

The PC build a fee-based email client, fee-based email server, per-user client fee, fee for email product maintenance, and fee for their proprietary to standards-based internet gateway. For some, using graphical and multimedia email from the late 1980's, for 5 years, and having to revert to Microsoft Mail on a PC in the early 1990's was terribly painful, but proverbial "bean counters" thought paying for a non multimedia based email with proprietary internet gateway on a PC based client-server systems was less expensive than free on an integrated UNIX desktop. With unified communication on a modern early 201x desktop, it is not unusual to receive voice mail over the computer, but one now has to pay for a VoIP PBX, VoIP clients to call (and leave voice messages), and pay client licenses on the servers side - for a capability which existed under Sun Workstations from the late 1980's. More on this, shortly.

Microsoft bundled Microsoft Mail client briefly with their desktop OS, which would only work against their proprietary email system. Netscape created one of the first commercial browsers with integrated email as well as integrated web and email server. Internet providers started becoming more common, funded and bundled the Netscape integrated web client. The dominate desktop vendor, after effectively defeating UNIX open desktop workstations with their proprietary Windows operating system, drove Netscape out of business by making it impossible for system vendors to bundle the popular Netscape client software, and by distributing a free web browser on their OS client. Eventually, Microsoft pulled their half-featured desktop client and started selling an integrated email client called Outlook.

Netscape was purchased by AOL, assets were split between Sun Microsystems for back-end software, while the Mozilla Foundation receive the integrated email and web clients, and AOL received Netscape's on-line internet presence.) The integrated Mozilla integrated web client was split into various components: Firefox (browser), Thunderbird (email), Sunbird (calendaring.) There was virtually no market to fund innovation for the desktop clients, with dominance of proprietary email back-ends and free desktop clients.

Major internet presence companies such as AOL, Yahoo, Google, and (late-comer) Microsoft built increasingly integrated email and calendaring environments in the ever more complex (and capable) HTML standard language. The need for desktop clients started to disappear, as long as there was a reasonable desktop web client. The funding for the next generation of email and calendaring interfaces did not originate with selling software.

The new competition for email clients is no longer other email clients. People are increasingly turning to proprietary email solutions on embedded devices (ex. iPhone, iPad, etc.) where the hardware (or telco service fees) provide funding and integrated web based email clients, who find their funding stream via advertizing revenue.

Thunderbird News:

The Mozilla foundation, who divested the integrated Netscape client into Thunderbird and Firefox, started getting increasing pressure from large commercial internet presence, Google, with the creation of Chrome and the massive resources injected into a fast development cycle for their own namesake web browser, Chrome, which was bundled on their own popular embedded devices.

With the enhancement of internet clients, increasingly being bundled in vertical channels (Outlook for Microsoft Exchange, Internet Explorer for Microsoft Windows, Chrome web browser for Android Phones/Tablets, Apple Email/Safari for MacOSX/iPhone/iPad) and increasing investment by internet content providers (i.e. Yahoo, Google, Microsoft, AT&T, Verizon, AOL) of increasingly powerful web mail - pressure seemed to be placed on the entire Mozilla Foundation. The following delineated some of the most recent news regarding Thunderbird.

• Mozilla's Thunderbird, as was published a few days back by tech crunch, will soon be moving to the back burner
• It is expected that Open Source developers will need to pick up the slack, as Mozilla developers will move to more pressing projects.
• A simple governance model is being prepared, to take care of Thunderbird, in the future.
• Thunderbird ESR, or Extended Support Release, will not necessarily be impacted. Commercial and Education institutions will continue to get their critical support.
• The chairman of the Mozilla foundation posted an official blog response regarding the move. Community members involved in localization have contributed significantly, but other community members seem to be happy with what they have... with many people moving to web browser or vendor bundled proprietary email clients.
• Some consider the move to be a death knell for Thunderbird, wondering where people will move to next.

Network Management:
Some may be asking - what is the opinion of the Network Management team. If Mozilla will continue to keep Thunderbird on modern releases of the browser rendering engine, it might not be that significant. It seems community involvement has been lacking and there has not been much of a driver for innovation. As long as Microsoft is selling an email client, Thunderbird remains relevant.

What is little mentioned was Netscape's and Mozilla's Thunderbird NNTP (network news transfer protocol) integration. Usenet's net news integration in Netscape and Thunderbird was one of the key collaboration features, which is supported by Thunderbird, from which there is no other client. There are various web based NNTP gateways, but the Thunderbird client support was very robust.

With security fixes continuing to happen every 6 months, there will still be work ahead for the network management community. If you are a Thunderbird user,  it appears your long term investment will be maintained, but please get involved if you are interested in newer features.

Clustered File Systems: EMC and Lustre

[EMC VNX HPC Appliance Series, courtesy The Register]

EMC, Lustre, and Massive Storage
EMC announced a VNX appliance with built-in Lustre clustered filesystem storage solution. In an article published by The Register:

The VNX HPC appliance includes a VNX5100 for metadata storage, a VNX7500 for object storage, object and metadata servers, the Terascala LustreStack software and InfiniBand connectivity.

Terascala happens to be a storage start-up located in Massachusetts, just like EMC. EMC, by the way, started shipping Intel based blade servers, pushing Cisco out of their initial partnership. EMC looks

Where is Lustre on Oracle Solaris?
Sun Microsystems purchased Lustre in 2007. Lustre was to be merged onto ZFS. Systems with 256 and fewer nodes could use QFS while 512 nodes would use Lustre. Different updates to OpenSolaris were made to facilitate Lustre integration.

In 2008, Terascala was inquiring with scalability enhancements under ZFS. Oracle purchased Sun and announced support for Lustre 2.x only on Oracle hardware. In 2010, the move of Lustre to Linux distributions such as SUSE seemed inevitable, as Oracle abandoned their support model, and other companies like Clusterstor offered support.

When will Oracle release Lustre with ZFS under Solaris? When will Oracle release native Lustre support on the Oracle storage, as EMC has done?

Open Alternatives
Operating System forks OpenIndiana on source code forks like Illumos, could offer companies such as Terascala, EMC, SI, Xyratex, and such another option: native ZFS merged with Lustre on a base OS which supports all standard protocols: iSCSI, FiberChannel, NFS, CIFS, etc.

This would also solve some of EMC's problems, with needing to find another partner for another cloud project (after dumping Cisco) - they could own the entire cloud, from the hardware (their own blades), to the firmware (VMWare), to the OS (Illumos distribution), to a file system fork (ZFS), cluster fork (Lustre), and all the protocols that go along with Illumos.

When will Illumos release Lustre support?