Joyent: Encapsulating Linux through Docker into a Zone

[Solaris 11 Launch image, courtesy Oracle]

Abstract:

Virtualization has been available in the UNIX OS world. The creation of users in a time sharing environment, to isolate executable threads from one another as well as protect files in an underlying file system started the journey. The creation of the Virtual File System, where disks could me mounted anywhere in a file system tree (instead of drive letter) revolutionized computing to allow those systems to grow in the shared environment! The creation of "chroot" so an application could run in it's own file system space, made an application "feel" like it is on a dedicated system. The merging of SVR4 into Solaris created a robust multi-processor infrastructure to host multi-user and  multi-tenant systems. The creation of Zones under SVR4 Solaris 10, further extrapolated the original concepts of the UNIX "chroot", isolating CPU, Memory, Users, Storage - effectively making a single instance of the Solaris OS truly multi-tenant. The creation of Branded Zones for Linux and Solaris came later, offering entire operating systems to be encapsulated under Intel and SPARC Solaris systems. Newer proprietary technologies continue to enter the horizon.

[Oracle Linux, courtesy Oracle]

The Linux Problem

People participating in the Linux ecosystem are interested in creating new raw environments,  isolated to their operating system under proprietary Intel processors, to supply a reasonable replacement for mature infrastructure. These replacements constitute very long efforts, which often never really get completed. Veterans understand the benefit of good engineering and can often take systems "to the next level." Vendors like Oracle had taken Linux, ran their applications on top of it, and supplied the patches necessary to keep Linux stable.

Joyent: Zones(KVM and Linux)

Former employees of Sun Microsystems continue to do the heavy lifting in the industry. Network Management wrote about Joyent's efforts to port KVM into Solaris Zones under their SmartOS, based upon Illumos. Illumos originated from Sun Microsystem's OpenSolaris project (which became the basis of Oracle's Solaris 11.)

[Solaris Zone/Container concept, courtesy former Sun Microsystems]

Joyent: Zones(Docker and Linux)

One might expect that Cloud companies who are obsessed with Virtualization like Joyent would continue their quest for a "better cloud". In 2015, Joyent released a presentation on the porting of Docker to encapsulate Linux into a Zone... using the same SmartOS based upon Illumos, which found it's roots in Sun Microsystem's OpenSolaris.

Docker and the Future of Containers in Production from bcantrill

For Joyent, The Cloud means chasing every container technology and integrating it into SmartOS, to give their customers choice, while simultaneously utilizing their infrastructure as efficiently as possible.

Conclusion

SVR4 UNIX and Sun Solaris developers have a long history of virtualization. The success story of Joyent in "Cloud" environments continues to lead the market in vision, taking things which were good but raw, and rolling them into mature facilities which continues to make the computing industry grow!

Tab Update: OpenSXCE May 2014 Distribution

Abstract: 

Sun Microsystems created OpenSolaris on May 8 in 2008 as an Open Sourced version of it's enterprise and managed services grade Solaris operating system. In 2010, Sun was acquired by Oracle, OpenSolaris project was closed, Community based OpenIndiana was born, and the Oracle Solaris Express binary release program was officially re-instantiated. Independent maintainer Martin Bochnig produced his own Intel and SPARC release of MartUX in 2006, produced the first (and last) SPARC release of OpenIndiana in 2012, and later forked (and maintaining) his own SVR4 package based SPARC and Intel distributions called OpenSXCE. The most current OpenSXCE release is now 2015.05... perhaps the first OpenSolaris based SPARC distribution which offers USB boot support!

[OpenSXCE  by Martin Bochnig]

OpenSXCE

Introduction:

OpenSXCE 2014.05 is an enterprise-class OpenSolaris based server and desktop oriented distribution. Options include: Live-DVD, Live-USB, Virtual Box, LDOM - with local or remote SVR4 package network repository options. Both SPARC and Intel continue to be made available. This may be the FIRST USB Bootable OpenSolaris based release in history! Many congratulations to for maintaining platform-independent OpenSolaris releases, originally promised by the Illumos community.

2014.05 Release

[http] Main Web Site with Downloads
[txt] 2015.04 Release Notes
[http] 2014.05 x86/x64 Live DVD
[http] 2014.05 x86/x64 Live USB
[http] 2015.05 x86/x64 Virtual Box
[http] 2015.05 x86/x64 SVR4 Repository
[http] 2014.05 SPARC Live DVD
[http] 2014.05 SPARC Live USB
[http] 2015.05 SPARC LDOM
[http] 2015.05 SPARC SVR4 Repository[email] Maintainer - Martin Bochnig

Solaris: Loopback Optimization and TCP_FUSION

Abstract:
Since early days of computing, the most slowest interconnects have always been between platforms through input and output channels. The movement from Serial ports to higher speed communications channels such as TCP/IP became the standard mechanism for applications to not only communicate between physical systems, but also on the same system! During Solaris 10 development, a capability to increase the performance of the TCP/IP stack with application on the same server was introduced called TCP_FUSION. Some application vendors may be unaware of safeguards built into Solaris 10 to keep denial of service attacks or starvation of the applications due to the high performance of TCP writers on the loopback interface.

Functionality:
Authors Brendan Gregg and Jim Mauro describe the functionality of TCP_FUSION in their book: DTrace: Dynamic Tracing in Oracle Solaris, Mac OS X, and FreeBSD.

Loopback TCP packets on Solaris may be processed by tcp fusion, a performance feature that bypasses the ip layer. These are packets over a fused fused connection, which will not be visible using the ip:::send and ip:::receive probes, (but they can be seen using the tcp:::send and tcp:::receive probes.) When TCP fusion is enabled (which it is by default), loopback connections become fused after a TCP handshake, and then all data packets take a shorter code path that bypasses the IP layer.

The modern application hosted under Solaris will demonstrate a significant benefit over being hosted under alternative operating systems.

Demonstrated Benefits:
TCP socket performance, under languages such as Java, may demonstrate a significant performance improvement, often shocking software developers!

While comparing java TCP socket performance between RH Linux and Solaris, one of my test is done by using a java client sending strings and reading the replies from a java echo server. I measure the time spent to send and receive the data (i.e. the loop back round trip).
The test is run 100,000 times (more occurrence are giving similar results). From my tests Solaris is 25/30% faster on average than RH Linux, on the same computer with default system and network settings, same JVM arguments (if any) etc.

The answer seems clear, TCP_FUSION is the primary reason.

In Solaris that's called "TCP Fusion" which means two local TCP endpoints will be "fused". Thus they will bypass the TCP data path entirely. 

Testing will confirm this odd performance benefit under stock Solaris under Linux.

Nice! I've used the command
echo 'do_tcp_fusion/W 0' | mdb -kw

and manage to reproduce times close to what I've experienced on RH Linux. I switched back to re-enable it using
echo 'do_tcp_fusion/W 1' | mdb -kw

Thanks both for your help.

Once people understand the benefits of TCP_FUSION, they will seldom go back.

Old Issues:
The default nature of TCP_FUSION means any application hosted under Solaris 10 or above will, by default, receive the benefit of this huge performance boost. Some early releases of Solaris 10 without patches may experience a condition where a crash can occur, because of kernel memory usage. The situation, workaround, and resolution is described:

Solaris 10 systems may panic in the tcp_fuse_rcv_drain() TCP/IP function when using TCP loopback connections, where both ends of the connection are on the same system. This may allow a local unprivileged user to cause a Denial of Service (DoS) condition on the affected host.

To work around the described issue until patches can be installed, disable TCP Fusion by adding the following line to the "/etc/system" file and rebooting the system: set ip:do_tcp_fusion = 0x0.

This issue is addressed in the following releases: SPARC Platform Solaris 10 with patch 118833-23 or later and x86 Platform Solaris 10 with patch 118855-19 or later.

Disabling TCP_FUSION feature is no longer needed for DoS protections.

Odd Application Behavior:
If an application running under Solaris does not experience a performance boost, but rather a performance degradation, it is possible your ISV is not completely understand TCP_FUSION or the symptoms of an odd code implementation. When developers expect the receiving application on a socket to respond slowly, this can result in bad behavior with TCP sockets accelerated by Solaris.

Instead of application developers optimizing the behavior of their receiving application to take advantage of 25%-30% potential performance benefit, some of those applications vendors chose to suggest disabling TCP_FUSION with their applications: Riverbed's Stingray Traffic Manager and Veritas NetBackup (4x slowdown.) Those unoptimized TCP reading applications, which perform reads 8x slower than their TCP writing application counterparts, perform extremely poorly in the TCP_FUSION environment.

Possible bad TCP_FUSION interaction?
There is a better way to debug this issue rather than shutting off the beneficial behavior. Blogger Steffen Weiberle at Oracle wrote pretty extensively on this.

First, one may want to understand if it is being used. TCP_FUSION is often used, but not always:

There are some exceptions to this, including when using IPsec, IPQoS, raw-socket, kernel SSL, non-simple TCP/IP conditions. or the two end points are on different squeues. A fused connect will revert to unfused if an IP Filter rule will drop a packet. However TCP fusion is done in the general case.

When TCP_FUSION is enabled for an application, there is a risk that the TCP data provider can provide data so fast over TCP that it can cause starvation of the receiving application! Solaris OS developers anticipated this in their acceleration design.

With TCP fusion enabled (which it is by default in Solaris 10 6/06 and later, and in OpenSolaris), when a TCP connection is created between processes on a system, the necessary things are set up to transfer data from the sender to the receiver without sending it down and back up the stack. The typical flow control of filling a send buffer (defaults to 48K or the value of tcp_xmit_hiwat, unless changed via a socket operation) still applies. With TCP Fusion on, there is a second check, which is the number of writes to the socket without a read. The reason for the counter is to allow the receiver to get CPU cycles, since the sender and receiver are on the same system and may be sharing one or more CPUs. The default value of this counter is eight (8), as determined by tcp_fusion_rcv_unread_min.

Some ISV developers may have coded their applications in such a way to anticipate that TCP is slow and coded their receiving application to be less efficient than the sending application. If the receiving application is 8x slower in servicing the reading from the TCP socket, the OS will slow down the provider. Some vendors call this a "bug" in the OS.

When doing large writes, or when the receiver is actively reading, the buffer flow control dominates. However, when doing smaller writes, it is easy for the sender to end up with a condition where the number of consecutive writes without a read is exceeded, and the writer blocks, or if using non-blocking I/O, will get an EAGAIN error.

So now, one may see the symptoms: errors with TCP applications where connections on the same system are experiencing slowdowns and may even provide EAGAIN errors.

Tuning Option: Increase Slow Reader Tolerance
If the TCP reading application is known to be 8x slower than the TCP writing application, one option is to increase the threshold that the TCP writer becomes blocked, so maybe 32x as many writes can be issued [to a single read] before the OS performs a block on the writer, from a safety perspective. Steffen Weiberle also suggested:

To test this I suggested the customer change the tcp_fusion_rcv_unread_min on their running system using mdb(1). I suggested they increase the counter by a factor of four (4), just to be safe.

# echo "tcp_fusion_rcv_unread_min/W 32" | mdb -kw
tcp_fusion_rcv_unread_min:      0x8            =       0x20

Here is how you check what the current value is.

# echo "tcp_fusion_rcv_unread_min/D" | mdb -k
tcp_fusion_rcv_unread_min:
tcp_fusion_rcv_unread_min:      32

After running several hours of tests, the EAGAIN error did not return.

Tuning Option: Removing Slow Reader Protections
If the reading application is just poorly written and will never keep up with the writing application, another option is to remove the write-to-read protection entirely. Steffen Weiberle wrote:

Since then I have suggested they set tcp_fusion_rcv_unread_min to 0, to turn the check off completely. This will allow the buffer size and total outstanding write data volume to determine whether the sender is blocked, as it is for remote connections. Since the mdb is only good until the next reboot, I suggested the customer change the setting in /etc/system.

\* Set TCP fusion to allow unlimited outstanding writes up to the TCP send buffer set by default or the application.
\* The default value is 8.
set ip:tcp_fusion_rcv_unread_min=0

There is a buffer safety tunable, where the writing application will block if the kernel buffer fills, so you will not crash Solaris if you turn this write-to-read ratio safety switch off.

Tuning Option: Disabling TCP_FUSION
This is the proverbial hammer on inserting a tack into a cork board. Steffen Weiberle wrote:

To turn TCP Fusion off all together, something I have not tested with, the variable do_tcp_fusion can be set from its default 1 to 0.
...
And I would like to note that in OpenSolaris only the do_tcp_fusion setting is available. With the delivery of CR 6826274, the consecutive write counting has been removed.

Network Management has not investigated what the changes were in the final releases of OpenSolaris or more recent  Solaris 11 releases from Oracle in regards to TCP_FUSION tuning.

Tuning Guidelines:
The assumption of Network Management is that the common systems administrator is working with well-designed applications, where the application reader is keeping up with the application writer, under Solaris 10. If there are ill-behaved applications under Solaris 10, but one is interested in maintaining the 25%-30% performance improvement, some of the earlier tuning suggestions below will provide much better help than the typical ISV suggested final step.

Check for TCP_FUSION - 0=off, 1=on (default)

SUN9999/root#   echo "do_tcp_fusion/D" | mdb -k
do_tcp_fusion:
do_tcp_fusion: 1

Check for TCP_FUSION unread to written ratio - 0=off, 8=default

SUN9999/root# echo "tcp_fusion_rcv_unread_min/D" | mdb -k
tcp_fusion_rcv_unread_min:
tcp_fusion_rcv_unread_min:      8   

Quadruple the TCP_FUSION unread to write ratio and check the results:

SUN9999/root# echo "tcp_fusion_rcv_unread_min/W 32" | mdb -kw
tcp_fusion_rcv_unread_min:      0x8            =       0x20

SUN9999/root# echo "tcp_fusion_rcv_unread_min/D" | mdb -k
tcp_fusion_rcv_unread_min:
tcp_fusion_rcv_unread_min:      32

Disable the unread to write ratio and check the results:

SUN9999/root# echo "tcp_fusion_rcv_unread_min/W 0" | mdb -kw

SUN9999/root# echo "tcp_fusion_rcv_unread_min/D" | mdb -k
tcp_fusion_rcv_unread_min:
tcp_fusion_rcv_unread_min:      0

Finally, disable TCP_FUSION to lose all performance benefits of Solaris, but keep your ISV happy.

SUN9999/root# echo "do_tcp_fusion/W 0" | mdb -kw

May this be helpful for Solaris 10 platform administrators, especially with Network Management platforms!

OpenSolaris Successor: OpenSXCE (Part 2)

[OpenSXCE Mascot with Illumos Logo]

Abstract:

Solaris, once the standard in workstations, became the standard for datacenter server. Sun Microsystems had open-sourced Solaris, referring to it as OpenSolaris. Oracle, purchased Sun, released Solaris 11 upon the OpenSolaris foundation, and closed OpenSolaris. Various forks were made from the OpenSolaris code, Illumos being one of the most popular across various commercial businesses, but only OpenSXCE was successful in creating multiplatform SPARC and Intel editions. As of May 2005, Martin made his second release. The first article discussed the history of OpenSXCE, the first release, the test platform, and second release using the text installer. This second article will discuss the GUI installer.

[Sun Microsystems UltraSPARC 60 Desktop]

Third Impression

The Test Platform

The Ultra 60 workstation with 2x Gigabytes of RAM, 2x 450 GHZ processor, and 1x DVD on-board, had it's video card repaired and a Gigabit Ethernet Card added - for the use of installing OpenSXCE 2013.05.

[SPARC International Logo]

The Installation Process

The OpenSXCE full install DVD booted reasonably well on the UltraSPARC platform, bringing up a desktop. By default, an command line was opened, but the terminal needed to be moved in order to reveal the Installation icon on the desktop. For new users, this might catch them by surprise.

The menus were well defined and the platform came up reasonably quickly. A simplification from the OpenIndiana installation included a single username & password combination. The root password could be changed later, after installation, using "sudo ksh".

I would have liked to have seen an option to mirror the root disks in the GUI, as the latest version of Solaris 10 offers.

The DHCP worked great, the platform picked up an IP Address after install, but only on my built-in 100 MBit hme0 card. The Gigabit card was not detected.

[SunRay running FireFox under SPARC Solaris 10]

Post Installation Process

Sampling Applications

In the Application Menu, it should be noted that Apache Open Office is bundled. The included version was 3.4.1. This is a welcome usability enhancement to any desktop! The writer applications seemed fairly snappy, even on my 10 year old desktop. I was copying large quantities of text from an xterm to the Open Office Writer application and it seemed to respond more quickly on my dual 450 MHz SPARC than similar operations using Microsoft Word on my Dell Laptop with a modern Quad-Core Intel processor. During this time, about 25% of the CPU was being consumed while the Open Office Writer application was scrolling to accept the text from the copy-paste buffer.

Firefox was also included in the Applications menu - version 21. It was not as snappy as Apache Open Office was, but it was usable, and worked fine on the Network Management blog. Normally, Firefox on a V240 over a SunRay is more snappy than this dedicated workstation. The Adobe Flash plugin was required (and not installed by default) - but somehow the system switched into HTML5 mode and played a video, albeit very slowly.

The System Tools Performance Meter is an abysmal CPU sucker! Once the Gnome tool is brought up, nearly half the CPU capacity on the dual-processor desktop is sucked up - scrolling graphs, which do not need to be updated nearly as quickly! When one clicks off the resource tab, the CPU time drops way down - the issue appears to be with the scrolling of the 3 graphs. I would suggest defaulting the poll & scroll to every 1-5 second, instead of multiple times a second. A tool to measure performance should never impact system performance.

The calculator was usable. A user could tap the keys with a mouse - even the numeric key pad on the Sun Type 4 Keyboard worked correctly.

[Protocol for remote desktop]

Sampling System Utilities

If the user did not already change the root password, this should be done via "sudo ksh" and using the "passwd root" command, before trying to access the standard system utilities. Many of these utilities require the root password.

When checking the System -> Preferences -> Display, it failed with "Could not get screen information" and a sub-error of "RANDR extension is not present". I suspect this has to do with running the the capabilities of "openXsun" that is required for the UltraSPARC system.

There was an interesting "System -> Preferences -> Desktop Sharing" application. When selecting the "Allow other users to view your desktop" - it very pleasantly described the IP Address which was now sharing. When trying to attach to the Ultra60 from my iPhone via a VNC Viewer, a pop-up appeared on the Ultra60 desktop requesting permission. After providing permission, it worked, but my iPhone app was not very good at controlling it.I tried "vncviewer" on my V240 over a SunRay - and it experienced the same problem of the commands going to the desktop but the remote viewer not reflecting the changes. A restart of the Ultr60 forced a disconnection, so I could be sure that I could run my next tests cleanly.

I brought up the System -> Services Application and thought the layout of the tool was very user friendly. I was surprised not to see the SNMP daemon in the list. This is something I was specifically looking for. The /usr/sbin/snmpd and /usr/sbin/snmptrapd binaries existed, but no default service available. This is, perhaps, the most important piece of network & systems management - yet it is the most overlooked piece, by most vendors.

Final Thoughts

This was an excellent release, so far. As more progress is made, from a networking perspective, it will be posted to the Network Management blog. Kudos to making OpenSXCE happen - I look forward to contributing to the project!

OpenSolaris Successor: OpenSXCE (Part 1)

[OpenSXCE Mascot with Illumos Logo]

Abstract:

Solaris, once the standard in workstations, became the standard for datacenter server. Sun Microsystems had open-sourced Solaris, referring to it as OpenSolaris. Oracle, purchased Sun, released Solaris 11 upon the OpenSolaris foundation, and closed OpenSolaris. Various forks were made from the OpenSolaris code, Illumos being one of the most popular across various commercial businesses, but only OpenSXCE was successful in creating multiplatform SPARC and Intel editions. As of May 2005, Martin made his second release.

[OpenSolaris Logo]

History of OpenSXCE

Sun used to distribute a Solaris Express Community Edition (SXCE) and Solaris Express Developers Edition (SXDE) - but when Sun was preparing to drive towards Imaging Packaging system, these were canceled. OpenSolaris's final release supported both SPARC and Intel. One of the early forks from OpenSolaris was Illumos. Their premier distribution was OpenIndiana. OpenIndiana promised to also support SPARC and Intel, but people waited for years. Martin Bochnig invested the time into releasing an initial SPARC release of OpenIndiana, called MartUX. The Illumos and OpenIndiana team did not integrate his work, the wiki pages associated with his effort were purged, but Martin eventually released OpenSXCE - a fully SVR4 compliant SPARC and Intel release! At first, there was 2013.01 release, next came 2013.05. Martin was encouraged to possibly rename his distribution, to remove his name, and choose a more fitting title - OpenSXCE was born.

First Impressions

Considering the complexity & effort, the first 2013.01 DVD was impressive. The booting on SPARC actually worked, but the disk was only a live DVD. Clearly, he was close, and the community was amazed that he was able to do what former SPARC engineers from Sun Microsystems, who found their way to Illumos & OpenIndiana, were unable to accomplish!

A Pause and Moving Ahead...

Personal concerns forced me to severely curtail my community activity, much of the SPARC test equipment was placed into storage. As responsibilities started to clear more recently, the resurrection of SPARC equipment resumed.

[Sun Microsystems UltraSPARC 60 Desktop]

The Test Beds

A Microsoft Windows XP laptop with Hyperterimnal became the serial console for all servers.

Populating V120's with 15K drives began. Initial installations of SPARC Solaris 10 Update 10 had started, to ensure the platforms were functional. A single slim-line CD-ROM drive on a V120 platform with 3 Gigabytes of RAM would make a reasonable test-bed for a CD based Text based installer.

An Ultra 60 workstation with 2 Gigabytes of RAM was running a multi-terabyte file service using flash to accelerate the ZFS performance, but it's video card drop dead about 1 year ago. The Ultra60 has 2 Gigabytes of RAM and a DVD on-board, so it would still be reasonable for a DVD installer test.

[Solaris Logo]

The control case: Solaris 10

Solaris 10 can no longer be installed from a CDROM, but only from a DVD drive. Many older Sun platforms do not have DVD drives, and some DVD drives which fit those platforms are not always recognized.

Solaris 10 Update 10 has a nice feature where identical drives can be installed as a mirrored ZFS root during installation. This functionality was thoroughly tested by pulling drives out of the V120 while running an application which was performing constant writing writing to the disks, and a second application which was performing constant reading from those same disks - no indication of any hardware issues were noted by the applications, except error reporting on the console. The applications did not experience any degradation, even when drives were re-inserted, and resync'ing.

The resync'ing process for ZFS drives in a broken mirror takes seconds for short breakage durations, in contrast to other platforms where re-syncing a broken mirror would take hours, or sometimes days with larger drives. The applications on the Solaris 10 platform continued to operate flawlessly. during the operation.

[ZFS: The Default File System for OpenSXCE]

Second Impressions: OpenSXCE and Text Install

Windows Client Setup

If the terminal selection was not set to VT100 from the start, or Hyperterm was not correctly set to use Vt100, then the GUI might not operate as expected. Once the terminal was properly set up to use VT100 and Hyperterm was supposedly set up to use VT100 emulation, The installation went quickly. Most of the most common defaults were pre-selected, to make installation much easier.

SCSI Disk Drive Format

One of the two SCSI disks were not formatted, so I needed to drop out to a command line and format the disk. The "format" command was well known by my, but most people who use IDE, ATA, SATA, and USB drives are probably not familiar with the process. The process to conduct the format would have been nice had it been documented, but honestly - I don't remember seeing any recent UNIX operating system take people through that process automatically during the install process. The format command indicated it would take a little over 1 hour to format the drive - it took close to 24 hours to format the 72 Gigabyte 15K drive! (I am glad I did not have a 900 Gigabyte SAS drive, since even a hardened user might have thought there was a problem when such a format could have taken a week!)

Installation Notes

The installation documentation and process inform the user that the mirroring of the ZFS hard drives could be done after installation. This is a nice touch, since Solaris 10 Update 10 does this from the start as an option in the installer. This gives the user an opportunity to provide parity with Solaris 10 Update 10, without adding the complexity to the installer.

Install Experience

The installation was extremely quick - much faster than what I was expecting! I walked away during the install and it was completed when I returned. I will watch during my next install to time it properly. This truly beat Solaris 10 Update 10 install, hands-down.

There is an option to review the installation log, at the end of the install. The Hyperterm vt100 emulator appears to have been lacking since the screen split into 2 sides, where the left hand side seemed to scroll correctly, but the right hand side did not. Die-hard UNIX administrators would know that the [control][l] could be used to perform a refresh when the text screen became corrupt. It would have been nice to have such a warning, for new-be's - but I have not honestly seen such warnings for many years with textual installers.

Post-Install Experience

Pressing the [F8] key from the Hyperterm emulator did not initiate a reboot, at the final screen, as expected. There was a warning during the installation that function keys occasionally do not work from the emulators, and a user could revert back to using [ESCAPE] with a numeric aterwards. This worked flawlessly, the reboot started immediately after pressing [ESCAPE] followed by an [8].

Another thing which caught me by surprise, the root user was not allowed to log in from the serial console after installation, but the additional user specified during install MUST be used. I may have missed this during the installation warnings, perhaps it should be more clearly noted on the install screens.

There were some odd errors noted, with an "ldom" service, which did not surprise me since I was running on older SPARC hardware which did not support LDom's (or more recently known as Oracle VM for SPARC.) This functionality should only work on modern processors.It would be nice to have a check for old hardware and silently exit, or at least indicate that this is expected behavior on this class of hardware.

Amazing to me, there was no problem with the installation coming up in a fully functional way, with networking operating the way it was supposed to. This was some nice attention to detail.

Full Internet-Based Upgrade

The root user automatically provided a note which offered to upgrade the text installation from an external internet based repository. This was a wonderful addition, being able to use System V Revision 4 standard packaging formats for upgrading the OS from the internet! People may have noted that Sun (and later Oracle) decided to move from SVR4 packaging to a proprietary packaging format to get this feature, when clearly moving to a proprietary format was never needed for this functionality.

The internet upgrade time was 1 hour and 55 minutes, which was provided by the SVR4 packaging system. This was very nice forethought by the distribution! The command to boot into the new boot environment was provided by the upgrade process - kudos again to the distribution! The instructions were not as clear as they could be, for the process to reboot into the new boot environment, for new users.

Everything was not perfect. I lost my network functionality after the network upgrade from the internet. The process for mirroring the root disks with ZFS was not as clean as I had hoped.

[Sun Microsystems UltraSPARC IV Systems]

Final Thoughts

My first impression of the new system can be summed up in a single word: impressed. How could a part-time programmer from Eastern Europe do what was promised and could not be delivered by former staff of Sun Microsystems? Truly, this is "hat's off" time to Martin, I hope OpenIndiana or Illumos take some of his work "side-stream" or "upstream", and someone hires him to work on Solaris based engineering project!

OpenIndiana: Build 151 Prestable 8 Released

[OpenIndiana Logo]

OpenIndiana Build 151 Prestable 8 Released

[OpenSolaris Logo]

Abstract:

Sun Microsystems had open-sourced it's Solaris operating system during the process of building Solaris 10. The open source project became known as OpenSolaris, with the first Intel & SPARC distributions being also known an OpenSolaris. Oracle purchased Sun and updates to OpenSolaris stopped. A fork called Illumos was created, containing some of the source code. OpenIndiana became well known with their oi_151a0 distribution. In August 2013, the OpenIndiana team had released their latest update: oi_151a8.

[OpenIndiana Build 147 Screenshot]

First Impressions:

The OpenIndiana & Illumos teams have been unable to deliver on their SPARC commitment, but they were able to get their 9th binary Intel based release out. The installation was done against a Gateway Desktop with 1.5Gig of RAM. A pair of dual enterprise grade 250 SATA drives were recognized. No option was noted during the installation to create a mirrored pair, as was available with the most recent Solaris 10 updates. The GUI installation was marvelous, installing on the 32 bit Pentium 4 platform. The hyperthreading normally available in the Intel Pentium 4 did not look like it was available, as a second virtual CPU. A USB keyboard and mouse from a SunRay was recognized without any problems. The network card was not recognized, making the distribution less useful that I hoped.

[OpenIndiana Build 147: Package Manager]

Final Thoughts

The inability to use the built-in Network Card on the Gateway was problematic. Honestly, the reason this platform was being spun up was to test some Network Management software, normally run on SPARC platforms. This will start the process of determining what can be done next, to build out a supportable Intel platform. Off to the mailing lists!

Updated Tab: OpenSXCE

Network Management updated the OpenSXCE Tab - New Release Information!

[OpenSXCE  by Martin Bochnig]

OpenSXCE

OpenSXCE 2013.05 is an enterprise-class OpenSolaris-based server and desktop-oriented Install and Live DVD distribution. OpenSXCE runs primarily on Intel CPU-compatible and Sun UltraSPARC sun4u/sun4v-compatible 32-bit & 64-bit computers. Both GNOME and IceWM are provided with major applications like Firefox 21.0, Thunderbird 17.0.6, GIMP 2.6.7, OpenOffice 3.4.1, Pidgin 2.6.5, Songbird, Ekiga, Oracle Solaris Studio 12.x professional C/C++ compiler, and selected GNOME-based desktop applications. Use Wine 1.4.1 for installing Windows-based applications like PhotoShop, Illustrator, Dreamweaver, Libreoffice, or even your favorite video game. OpenSXCE is indirectly based upon portions of Illumos.
2013.05 and Newer Releases, Based Upon DilOS

[http] Main Web Site with  Downloads
[http] Official OpenSXCE Blog
[http] 2013.01 Live SPARC DVD
[http] 2013.05 Live x86 DVD
[http] Twitter for latest comments
[http] Maintainer - Martin Bochnig

New Tab: DilOS!

Network Management has introduced a new Source Code Repository and Distribution Tab: DilOS

DilOS is focused to be a development environment for illumos and userland builds. It supports build env with DPKG and IPKG zones. It specifically targets Intel platforms, but additional distributions are being hosted for other processors like SPARC.

• DilOS can be a base platform for new distros - to create new APT repo's with DEB packages and ISO's.
• DilOS can be installed to a Virtual machine (such as: VMware, VBox, etc) or bare metal with text console and SSH access.
• DilOS contain: dilos-userland + dilos-illumos-gate + converted binaries to DEB packages from OpenIndiana (oi-experimental).

[http] Introduction to DilOS

[http] Frequently Asked Questions

[http] Documentation

[http] Download

Tab Update: OpenSXCE March 2013 Distribution

[OpenSXCE  by Martin Bochnig]

Added to Network Management - a new tab: OpenSXCE !

OpenSXCE

OpenSolaris grew from an Open Source repository to Open Solaris Distribution (for both Intel and SPARC.) Solaris Express Community Edition (Solaris SXCE) was the Intel/SPARC forerunner of Oracle Solaris 11, which abandoned UltraSPARC processors. OpenSXCE, based upon the work of MarTUX, brings OpenIndiana and Illumos back to SPARC as a full distribution, based upon standards such as SVR4 packaging.

[http] OpenSXCE and MartUX Information (Hosted By: OpenIndiana)
[http] Main Distribution Web Site (Warning: No-Frills)
[email] Maintainer - Martin Bochnig
[iso] OpenSXCE 2013.01 Live SPARC DVD ISO
[http] SVR4 Package Repository

MartUX: Tab Updated!

For those of you waiting for consolidated information on the SVR4 SPARC OpenIndiana iniative - MartUX now has a Network Management Tab!

A copy of it is now found here:

---

MartUX is Martin Bochnig's SVR4 release of OpenIndiana on SPARC. MartUX was based upon OpenSolaris. Originally released only on sun4u architecture, it now also runs as a Live-DVD under sun4v.

[http] MartUX Announcement: September 27, 2012
[http] MartUX OpenIndiana 151 Alpha Live-DVD Wiki
[http] Martin Bochnig's Blog for MartUX, OpenIndiana SPARC port from OpenSolaris
[http] Martin Bochnig Twitter account
[http] MartUX site (down since storage crash)
[http] MartUX site snapshot from archive.org
[html] Network Management MartUX articles

TribbliX: Tab Updated!

For those waiting for consolidated information - TribbliX now has a consolidated tab on Network Management blog!

A current snapshot can be seen here:

---

Tibblix is Peter Tribble's incarnation of an SVR4 based OS, leveraging Illumos at the core. This is an Intel only distribution at this point. Tribblix is a brand-new pre-release, downloaded as a live-dvd, and based upon OpenIndiana.

[html] Tribblix Announcement: October 23, 2012
[html] Distribution Home Page
[html] Download Page
[html] Installation Guide
[html] Basic Usage Starting Pointers
[html] Peter Tribble Blog
[html] Peter Tribble Solaris Desktop Blog
[html] Network Management Tribblix Posts

MartUX: Second SVR4 SPARC Release!

MartUX History: MarTux is an OpenSolaris based desktop, created by Martin Bochnig, and released for SPARC platforms. It was originally released in October of 2009, billed as the first distribution on SPARC, but only on sun4u architectures. An early screenshot is here. Some of the early work proved difficult in compressing OpenSolaris onto a single live DVD, but with the help of the BeliniX community, the work was successful. A sun4v release would soon follow.

MartUX - The New Work:
In July 2012, Martin started his new work, of updating MartUX on SPARC. In August, Martin indicated he was "working day and night" to continue progress on a new distribution cut. The question was broached on the SPARC distribution again, in September. The shift from IPS to Live-DVD occurred, in order to release a product. People discuss buying of older SPARC hardware and reserving some equipment, in preparation for the new distribution. Martin posts some screenshots of the new distribution on September 10. Discussions about under 10% savings between gzip-5 and gzip-9 occur, and the drawback of great CPU usage during the packaging phase. By September 27, a MartUX DVD ISO was made available.

MartUX - September 2012 Release:
By the end of September of 2012, there was an indication a new distribution based upon OpenSolaris code base and OpenIndiana. OpenIndiana now uses a new living codebase of Illumos to augment the static code base of OpenSolaris. The OpenIndiana community had screenshots and instructions noted on their 151a release wiki of the MartUX LiveDVD. A short, but useful articles appeared on some of the industry wires such as Phoronix. MartUX OpenIndiana SPARC release is also now booting on the first sun4v or newer "T" architectures from Sun/Oracle!

Tragedy Strikes:
October 4 hits, Martin starts the process of formalizing the distribution. The goal is to move from his previous MartUX distribution name. Martin, however, had a catastrophic disk failure. His struggling was noted on a different list, but he comments about it on the OpenIndiana list, suggesting using triple-mirrors, and disconnecting the third mirror to ensure a solid backup.

Where's Martin?
Martin mentioned on October 2nd through twitter that Dave Miller was working on an installer and pkgadd. There was little sign of Martin posting in November on the OpenIndiana list, after his hard disk issues, but an occasional tweet here or there. Martin set up a separate blogspot for SPARC OI work. The martux.org web site is down, as of the end of November, although a snapshot of the content is still available on archive.org. We hope all is well, as he continues his unbelievable work on a SPARC release.

Tribblix Pre-Release!

A new offspring of the OpenSolaris family is rising: Tribblix!

Announced as a live-dvd by Peter Tribble on the OpenIndiana discussion list, let's see how this new SVR4 packaged system moves along!

Note, the new tab on Network Management.

---

Tibblix is Peter Tribble's incarnation of an SVR4 based OS, leveraging Illumos at the core. This is an Intel only distribution at this point. Tribblix is a brand-new pre-release, downloaded as a live-dvd, and based upon OpenIndiana.


[html] Announcement: October 23, 2012
[html] Distribution Home Page
[html] Download Page
[html] Installation Guide
[html] Basic Usage Starting Pointers
[html] Peter Tribble Blog

Solaris Services

Abstract:
Services in the System VR4 world were traditionally managed through facilities such as /etc/rc scripts, /etc/initab, and /etc/inetd.conf family of facilities. There were disadvantages regarding these families, regarding configuration differences, monitoring for failures, and provisioning. Solaris 10 introduced SMF, or Solaris Service Management Facility, which is now available in most modern operating systems. This script helps to recursively display the dependent services.

Script:
The script must remain in your execution path and must have the executable bit set.

#!/bin/ksh
# script: r_svcs2.sh
# author: david halko
# license: cddl - must retain authorship note in any usage
# purpose: recursively show dependent services maximum level of 4 deep
# version: 1.0 - released using + instead of tabs for blogspot illustration
# usage: r_svcs2.sh [service]
# examples: r_svcs2.sh inetd
# r_svcs2.sh gdm2-login
#
Param="${1}"
Deep="${2}"

[ "${2}" = "++++" ] && exit # exit if too many levels deep

if [ "${1}" = "" ]; then
nawk '/^#/ !/^#/ { exit }' `whence ${0}`
else
echo "${Deep}\c"
svcs -H ${Param} 2>&1 || exit
svcs -dH ${Param} 2>&1 | nawk '{ print $3 }' | sort -u | while read Param2
do
r_svcs2.sh "${Param2}" "+${Deep}"
done
fi

Help Screen:
The help screen will come up as default if you do not supply any options.

V240/user$ r_svcs2.sh
#!/bin/ksh
# script: r_svcs2.sh
# author: david halko
# license: cddl - must retain authorship note in any usage
# purpose: recursively show dependent services maximum level of 4 deep
# version: 1.0 - released using + instead of tabs for blogspot illustration
# usage: r_svcs2.sh [service]
# examples: r_svcs2.sh inetd
# r_svcs2.sh gdm2-login
#

Example Run:
The expanded dependencies can get quite large. An arbitrary number of 4 levels deep was coded into the script to keep the script from running for hours on slow machines. A simple run is illustrated below.

V240/user$ r_svcs2.sh gdm2-login
disabled Sep_29 svc:/application/gdm2-login:default
+online Sep_29 svc:/system/filesystem/local:default
++online Sep_29 svc:/milestone/single-user:default
+++online Sep_29 svc:/milestone/devices:default
+++online Sep_29 svc:/milestone/network:default
+++online Sep_29 svc:/network/loopback:default
+++online Sep_29 svc:/system/cryptosvc:default
+++online Sep_29 svc:/system/filesystem/minimal:default
+++online Sep_29 svc:/system/identity:node
+++online Sep_29 svc:/system/installupdates:default
+++online Sep_29 svc:/system/keymap:default
+++online Sep_29 svc:/system/manifest-import:default
+++disabled Sep_29 svc:/system/metainit:default
+++online Sep_29 svc:/system/patchchk:default
+++online Sep_29 svc:/system/sysevent:default
++online Sep_29 svc:/system/filesystem/minimal:default
+++online Sep_29 svc:/system/device/local:default
+++online Sep_29 svc:/system/filesystem/usr:default
+online Sep_29 svc:/system/utmp:default
++online Sep_29 svc:/milestone/sysconfig:default
+++online Sep_29 svc:/milestone/single-user:default
+++online Sep_29 svc:/system/sysidtool:net
+++online Sep_29 svc:/system/sysidtool:system

EMC: Building The Cloud, Kicking Cisco Out?

EMC: Building The Cloud, Kicking Cisco Out?

Abstract:
EMC used to be a partner in the Data Center with a close relationship with vendors such as Sun Microsystems. With the movement of Sun to create ZFS and their own storage solution, the relationship was strained, with EMC responding by suggesting the discontinuance of software development on Solaris platforms. EMC purchased VMWare and entered into a partnership with Cisco - Cisco produced the server hardware in the Data Center while EMC provided VMWare software and with EMC storage. The status-quo is poised for change, again.

 

[EMC World 2012 Man - courtesy: computerworld]

EMC World:
Cisco, being a first tier network provider of choice, started building their own blade platforms, entered into a relationship with EMC for their storage and OS virtualization (VMWare) technology. EMC announced just days ago during EMC World 2012 that they will start producing servers. EMC, a cloud virtualization provider, a cloud virtual switch provider, a cloud software management provider, a cloud storage provider, has now moved into the cloud server provider.

Cisco Response:
Apparently aware of the EMC development work before the announcement, Cisco released FlexPods with NetApp. The first release of FlexPods can be managed by EMC management software, because VMWare is still the hypervisor of choice. There is a move towards supporting HyperV, in a future release of FlexPods. There is also a movement towards providing complete management solution through Cisco Intelligent Automation for Cloud. Note, EMC's VMWare vCenter sits as a small brick in the solution acquired by Cisco, including NewScale and Tidal.

 

[Cisco-NetApp FlexPod courtesy The Register]

NetApp Position:
NetApp's Val Bercovici, CTO of Cloud, declares "the death of [EMC] VMAX." Cisco has been rumored to have been in a position to buy NetApp in 2009, 2010, but now with EMC marginalizing Cisco in 2012 - NetApp becomes more important, and NetApp's stock is dropping like a stone.

[former Sun Microsystems logo]

Cisco's Mishap:
Cisco, missing a Server Hardware, Server Hypervisor, Server Operating System, Tape Storage, Disk Storage, and management technologies, decided to enter into a partnership with EMC. Why this happened, when system administrators in data centers used to use identical console cables for Cisco and Sun equipment - this should have been their first clue.

Had Cisco been more forward-looking, they could have purchased Sun and acquired all their missing pieces: Intel, AMD, and SPARC Servers; Xen on x64 Solaris, LDom's on SPARC; Solaris Intel and SPARC; Storage Tek; ZFS Storage Appliances; Ops Center for multi-platform systems management.

Cisco now has virtually nothing but blade hardware, started acquiring management software [NewScale and Tidal]... will NetApp be next?

 

[illumos logo]

Recovery for Cisco:
An OpenSolaris base with hypervisor and ZFS is the core of what Cisco really needs to rise from the ashes of their missed purchase of Sun and unfortunate partnership with EMC.

From a storage perspective - ZFS is mature, providing a near superset of all features offered by competing storage subsystems (where is the embedded Lustre?) If someone could bring clustering to ZFS - there would be nothing missing - making ZFS a complete superset of everything on the market.

Xen was created around the need for OpenSolaris support, so Xen could easily be resurrected with a little investment by Cisco. Cloud provider Joyent created KVM on top of OpenSolaris and donated the work back to Illumos, so Cisco could easily fill their hypervisor need, to compate with EMC's VMWare.

 

[SmartOS logo from Joyent]

SGI figured out they needed a first-class storage subsystem, and placed Nexenta (based upon Illumos) in their server lineup. What Cisco really needs is a company like Joyent (based upon Illumos) - to provide storage and a KVM hypervisor. Joyent would also provide Cisco with a cloud solution - a completely intregrated stack, from the ground on up... not as valuable as Sun, but probably a close second, at this point.