Thursday, December 29, 2011

Solaris 11: A Cloud in a Box!

Solaris 11: A Cloud in a Box!
Abstract:
Computing industry began with resource centralized on singularly large computing platforms. The microprocessor brought computing power into the hands of individuals in homes and offices, but information was still centralized in each location. The creation of The Internet allowed for the sharing of information between homes and offices, around the globe. Reliable server and telecommunications infrastructure was required to make it work, applications were somewhat limited to a handful of standard Internet protocols, such as HTTP. Cloud Computing has been coming of age over the past number of years, driving custom applications to proprietary API's, to move more applications into the Internet, but this is quickly changing as operating system vendors include more robust virtualization. Cloud Computing is really about the virtualization of Internet infrastructure, including servers, to a point where pieces do not have to reside on the internet, nor in an office, nor just a split between the two - but can reside anywhere, including entirely in a laptop. Solaris 11, the first Cloud Operating System, offers the ability to virtualize everything, from entire data centers across thousands of platforms, to thousands of platforms virtualized on a laptop.

Simulating The Cloud: A Practical Example

Joerg M., an Oracle employee and publisher of C0T0D0S0, discusses Solaris 11 with some of it's features, demonstrates the building of a cluster of virtual data centers within a single operating system instance. If someone runs a data center, they should consider reviewing the article to better comprehend the capabilities of what a "Cloud" could and should be.

It should be noted that "simnet" clause to the "create-simnet" and "modify-simnet" are formally undocumented, but documented in the OpenSolaris released source code, and leveraged in various other derived Open Source branches. One of the most important distributions being the Joyent SmartOS cloud operating system distributions.

Not Included, but Not Out Of Scope

What is not included in Joerg's example are actual systems on the edges of the cloud. Adding them is actually more trivial than adding the virtual routers which were created. Add virtual interfaces, virtual systems, databases to virtual systems, middleware to virtual systems, applications to virtual systems, add bandwidth & latency limitations to WAN links, add port limitations to virtual firewalls, etc.

Why Go Through the Exercize?

Once someone builds the entire datacenter "in the box", creation of the real data center becomes trivial. But why does this matter?
  • For the first time, real test environments can be simulated, soup-to-nuts, in an inexpensive way. There is no charge for virtualization in a Solaris world.
  • Costs can be reduced by placing all development systems into a couple of "clouds" for virtually any number (Solaris supports over 4000 zones on a single OS instance) of applications
  • Movement of an application from development to test is as easy cloning a Zone and instantiating the Zone on a Test platform.
  • Costs can be reduced by placing all test systems into a couple of clouds for virtually any number of applications
  • Deploying tested application is as easy as instantiating the cloned test Zone on a production system
  • Disaster recovery is as easy as instantiating the Zone on the dead physical system onto a physical system in an alternate data center.
  • Deploying production applications into a cloud is as easy as backing up the application and restoring it into the cloud - not to mention bringing it back.
  • The interactions of the application with Firewalls, WAN's and LAN's are all well understood, with everything being properly developed and tested, making each production deployment seamless
The effort, with a step-by-step process will ensure that there are no missed steps in the process to bringing virtualization to a business.

Implications to Network Management

The world is slowly exiting the physical world and Network Management is no longer about monitoring edge routers and links - it is about monitoring virtualized infrastructure. Orchestration is all about automated deployment and cloud providers are getting better at this. The missing piece to this puzzle is robust SNMP management of everything. The creation of network management infrastructure needs to happen in the development clouds first, then the test clouds, so when the jump to production is complete - the management infrastructure has already been simultaneously developed and tested, with the applications.

Tuesday, December 27, 2011

JavaScript Tab Update

JavaScript Tab Update

Abstract:
JavaScript, formerly known as LiveScript, standardized as ECMAScript, is a language originally used on both client and server web platforms. Through unfortunate historical vendor interactions, server side usage of JavaScript became less common. With the advent of JavaScript engines, which can be decoupled from the client browser, JavaScript became usable on the server side for independent projects.

History:
Netscape Communications brought to the market some of the first widespread adopted web client http browsers and web servers. One of Netscape's key technologies was called LiveScript - a client and server side technology to bring automation and communication to the browser and server suite. With the advent of Sun Microsystems cross-platform Java language, Java quickly became a hit, and LiveScript was rebranded JavaScript. Microsoft soon released their own web client & server platform, but only included a somewhat compatible client side JavaScript, creating a proprietary language on the server-side, leaving JavaScript to become less common on the server. Nations quickly figured out that Microsoft's half-baked implementation was a bad for the world and standardization soon occurred through ECMAScript. Sun's Java, with it's cross-platform capability, quickly became the language of choice on the server while JavaScript became the language of choice on the client. The battle for the fastest web browsers created teams of developers building JavaScript Engines, which could be decoupled from the Web Client. With the advent of decoupled JavaScript engines, developers started the movement back to server side JavaScript. NodeJS is a recent server side JavaScript non-blocking framework. NodeJS is based upon Google V8 engine, which unfortunately only works on a subset of known servers architectures and operating systems.

Resources:
The following are recent resource changes to the JavaScript tab on the Network Management blog.

NodeJS Specific Developments
[html] Server side JavaScript Engine: Node.JS
[html] Community Support for NodeJS
[html] Internal Developers List for NodeJS
[html] X11 Client Implementation under NodeJS
[html] X11 "nwm" window manager
[html] XCB directly rendered by node-canvas
[html] Google V8 JavaScript Engine

JavaScript Engines
[html] Mozilla SpiderMonkey (Various platforms)
[html] Mozilla Tamarin (Various platforms)
[html] Mozilla Rhino (Java based JavaScript engine)
[html] Google V8 JavaScript Engine (Intel, ARM)
[html] Mozilla JaegerMonkey
[html] Apple WebKit Nitro (SquirrelFish Extreme)
[html] Opera Presto

Wednesday, December 21, 2011

Solaris 10: SSH and Forwarding HTTP


Solaris 10: SSH and Forwarding HTTP

Abstract:
When Sun first produced systems, the common way for users to move around a network and to distribute workload was to leverage the Berkeley "r" tools, such as "rsh", "rlogin", "rexec", etc. under Solaris. As academics became professional, security concerns over passwords being passed in the clear were raised and SSH was born. SSH was built with a compatible superset to "rsh", but this was later removed with the second version of the protocol. This document discusses the implementation of SSH under Solaris.

Global Configurations

SSH uses several global configuration files, one for the client, and another for the server. Each of these config files document the default compiler flags under Solaris. The "ssh" client global configuration file can be tailored on a per-user basis while the "sshd" server global configuration file is managed at the global level.

SSH Server Daemon

Under Solaris 10, related OS's, and above - SSHD is started through the services infrastructure.

sunserver/user$ svcs ssh
STATE STIME FMRI
online Aug_17 svc:/network/ssh:default
There are built-in compiled defaults and global defaults which are reviewed, upon startup, and connection.

Start a Session with X and HTTP Forwarding

For demonstration purposes, there may be the need to temporarily open an X Console (to install an Oracle Database) and forward HTTP ports (to test an application) on a platform in a DMZ. The sample command may look like this:

sunclient/user$ ssh user@sunserver -b 10.1.2.3 \
-L 58080:127.0.0.1:58080 -L 8080:127.0.0.1:8080 -g
Since the ports to be forwarded are over 1024, there is no requirement for special "root" permissions. The proxied HTTPD connections can be observed.

sunclient/user$ netstat -an  grep 8080
*.58080 *.* 0 0 49152 0 LISTEN
*.8080 *.* 0 0 49152 0 LISTEN
To perform a basic test of the forwarded HTTP port, the classic "telnet" can be used on the command line, but the connection is closed.

sunclient/user$ telnet localhost 58080
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connection to localhost closed by foreign host.
Note, the error on the remote side.

channel 5: open failed: administratively prohibited: open failed
This is a configuration issue.

Global SSHD Configuration

Under Solaris 10, forwarding agent is disabled as a compile flag, and is documented in the global configuration file. If one makes a connection via SSH, and proxies a port - an error message will be produced upon the first connection attempt to the proxied port.

To allow for the port forwarding, edit the configuration file "/etc/ssh/sshd_config".

AllowTcpForwarding yes
GatewayPorts yes
X11Forwarding yes
You will need to restart the "sshd" service, the administrative message disappears.

sunserver/root# svcadm restart ssh

Your port HTTP and X Windows Port Forwarding will now work for ad-hoc tasks.


Tuesday, December 20, 2011

Solaris Tab - Secure Deployment of LDom's or VM Server for SPARC


Solaris Tab - Secure Deployment of LDom's or VM Server for SPARC

An Oracle White Paper, Secure Deployment of Oracle VM Server for SPARC , was added to the Solaris Tab on Network Management.

Solaris Reference Material
2011-01 [PDF] Secure Deployment of LDom's or VM Server for SPARC

Solaris LDoms / Oracle VM Server for SPARC
Secure Deployment of LDoms or Oracle VM Server for SPARC

Monday, December 19, 2011

SPARC T4: Optimizing with Oracle VM Server for SPARC


SPARC T4: Optimizing with Oracle VM Server for SPARC

Abstract:

Modern computing systems had found their footing through the history of computing. Some companies and architectures influenced the modern desktop computer more than others. One such company was Sun Microsystems, which had found it's way into Oracle. Oracle released their latest processor, the SPARC T4, with a dynamic new capability to offer the functionality to process two different workloads, via virtualization technology.

Processor History:

In 1985, Sun Microsystems produced their first Sun-3 workstation and servers based upon the 32 bit CISC Motorola 68000 processor. In 1987, Sun Microsystems produced their first Sun-4 workstations and servers upon the 32 bit RISC SPARC processor. In 1995, Sun Microsystems produced their first UltraSPARC system based upon 64 bit RISC UltraSPARC processor. In 2002, Sun Microsystems acquired Afara Web Systems, with a new high-throughput SPARC design. In 2005, Sun Microsystems released their first server (no desktops) based upon the UltraSPARC T1 processor, which was tuned for multi-threaded workloads. Oracle, who made their fortunes primarily from software upon SPARC, acquired Sun Microsystems and released their first server (no desktops) in 2010 based upon the SPARC T3. Oracle released the SPARC T4 in 2011, supporting both multi-threaded and single-threaded workload.


Workload History:

The workloads in the SPARC processors were traditionally single-threaded workloads from their early years. With the advent of RISC processors, the concept to reduce complexity allowed for the increase clock speed and thus the increase of single threaded performance. With the investment from AT&T and merger with SVR4, Solaris experienced multi-threaded workloads expansion. When SGI purchased Cray Research, Sun Microsystems purchased the Cray Superserver 6400 to create massive high-speed single threaded capability into massive multi-threaded workload throughput of 64 threads via racks of equipment.

With the release of UltraSPARC T1, Sun Microsystems managed to shrink 32 threads of slower integer and crypto capacity not only into a single socket, but onto single piece of silicon, performing outstanding aggregate capacity. With the subsequent release of the T2 processor, 64 threads were merged onto a chip. While the throughput was equivalent to racks of equipment in the T processors, the single threaded performance was a decade behind.

Workload Selection:

With the release of the Oracle SPARC T4 processor, a system can now be tuned to support single or multi-threaded workloads via Oracle VM Server for SPARC release 2.1, previously known as Logical Domains or LDom's.

The short tuning white paper from Oracle describes:
This paper describes how to use the Oracle VM Server for
SPARC 2.1 CPU threading controls to optimize CPU performance
on SPARC T4 platforms. CPU performance can be optimized for
CPU-bound workloads by tuning CPU cores to maximize the
number of instructions per cycle (IPC). Or, CPU performance
can be optimized for maximum throughput by tuning CPU cores
to use a maximum number of CPU threads. By default, the CPU
is tuned for maximum throughput
During the provisioning of a Logical Doman or VM under SPARC, the provisioner can choose the workload optimization required. This can be performed during ["add-domain"] or after ["set-domain"] provisioning.
ldm add-domain [mac-addr=num] [hostid=num]
[failure-policy=ignorepanicresetstop]
[extended-mapin-space=on]
[master=master-ldom1,...,master-ldom4]
[threading=max-throughputmax-ipc] ldom

ldm set-domain [mac-addr=num] [hostid=num]
[failure-policy=ignorepanicresetstop]
[extended-mapin-space=[onoff]]
[master=[master-ldom1,...,master-ldom4]]
[threading=max-throughputmax-ipc] ldom
The "threading" parameter defines the workload. The options from the white paper are defined as follows:



  • max-throughput.
    Use this value to select the threading mode that maximizes throughput. This mode activates all threads that are assigned to the domain. This mode is used by default and is also selected if you do not specify any mode (threading=).

  • max-ipc.
    Use this value to select the threading mode that maximizes the number of instructions per cycle (IPC). When you use this mode on the SPARC T4 platform, only one thread is active for each CPU core that is assigned to the domain. Selecting this mode requires that the domain is configured with the whole-core constraint.

Sunday, December 18, 2011

Solaris Tab - SPARC T4 Workload Optimization


Solaris Tab - SPARC T4 Workload Optimization

A new Oracle White Paper, Tuning the SPARC CPU to Optimize Workload Performance on SPARC T4, was added to the Solaris Tab on Network Management.

Solaris Reference Material
2011-09 [PDF] Tuning to Optimize Workload Performance on SPARC T4

Friday, December 16, 2011

Oracle Ops Center 11g Release 1 Update 3

Oracle Ops Center 11g Release 1 Update 3

Abstract:
Datacenters have long struggled with the lifecycle management of servers on a massive scale. Sun Microsystems addressed this concern with their N1 product line, which was later re-branded xVM with additional consolidation of hypervisor. With the acquisition of Sun by Oracle, hypervisors were broken out and Ops Center has been placed under the umbrella of Oracle Enterprise Manager.

Ops Center History:
Ops Center has a long history with features consolidated from many startups and industry players, now conslidated under Oracle.
2001-10-26 [html] Terraspring Startup
2002-09-19 [html] Pyrus acquisition announced
2002-11-02 [html] Sun acquired Pyrus for virtualization
2002-11-15 [html] Sun acquires Terraspring for heterogeneous system automation
2003-07-03 [html][html] Sun acquired CenterRun for application automation
2003-12-04 [html] Sun releases N1 Service Provisioning System
2005-05-03 [html] Sun augments N1 Provisioning System with N1 System Manager
2007-11-16 [html] First Internet Archive OpenxVM.org capture
2007-12-04 [html] Sun announces xVM Ops Center and open-sourcing to OpenxVM.org
2008-05-28 [html] Sun xVM Ops Center 1.1.1 GA
2009-01-27 [html] Sun xVM Ops Center 2.0 GA
2009-02-27 [html] Final Internet Archive OpenxVM capture
2010-01-22 [html] Oracle xVM Ops Center 2.5.0.1171 GA

Upcoming Release:
Oracle Enterprise Manager Op Center 11g Release 1 Update 3 is about to be released. The upgrade documentation is now available, packages are soon to follow.

Thursday, December 15, 2011

From SunOS through Solaris to Illumos

From SunOS through Solaris to Illumos

Don't miss this slide show from Joyent

Wednesday, December 7, 2011

UNIX/Linux Vocabulary Building

The UNIX/Linux environment is a rich collaboration of tools, tricks, and jokes built by generations of users with widely varying levels of ability. While basic competence is achievable within a short period of time (i.e. "Just 5-10 years to learn the rules and only a couple lifetimes to master.") it's easy to become dependent on a few commands when other interesting or more suitable tools are readily available:

# find / | grep ifconfig
vs
$ whereis ifconfig


$ man ls
vs
$ pinfo ls

In this spirit I recommend the following links (not surprisingly, Dave's favorite AWK is listed in both).

Reddit thread:
Give Me That One Command You Wish You Knew Years Ago

Beware the spelling errors:
Advanced Unix Commands

Tuesday, December 6, 2011

Revisited: Oracle Database Licensing

Revisited: Oracle Database Licensing

Abstract:
Oracle licenses it's RDBMS by several factors, typically the Standard License (by socket) and an Enterprise License (by core scaling factor.) Occasionally, hardware and operating system vendors will enhance their offerings, requiring a revisit by database vendors to expand their legal categorizations for licensing. Oracle's guiding documents are readily available on-line.

Reason for Revisit:
Sun had produced several virtualization technologies, by the time Oracle purchased them. One particular virtualization technology, "LDoms" (short for Logical Domains), renamed to "Oracle VM for SPARC", has been added to the list of being approved for Physical Partitioning technologies.

Partitioning - Topic: Server/Hardware Partitioning
The Oracle Partitioning guide now approves of LDoms or Oracle VM for SPARC as a Hard Partitioning technology.
Oracle has deemed certain technologies, possibly modified
by configuration constraints, as hard partitioning, and no
other technology or configuration qualify. Approved hard
partitioning technologies include: Dynamic System Domains
(DSD) -- enabled by Dynamic Reconfiguration (DR), Solaris 10
Containers (capped Containers only), LPAR (adds DLPAR with
AIX 5.2), Micro-Partitions (capped partitions only), vPar,
nPar, Integrity Virtual Machine (capped partitions only),
Secure Resource Partitions (capped partitions only), Static
Hard Partitioning, Fujitsu’s PPAR, Oracle VM Server for SPARC.
Oracle VM Server for x86 can also be used as hard partitioning
technology only as described in the following document
Implications for Network Management:

With the current SPARC T4 systems, this becomes more important for Managed Services environments, where Service Provider licenses are required in order to perform external services with an Oracle RDBMS. Being able to limit the number of cores on a new quad socket SPARC T4-4 system offers a lot of flexibility - especially when performance characteristics are similar to 8 socket POWER7 and 32 socket SPARC64 VII platforms.

Most network management software is available under SPARC and few are available under POWER, yet there has been a movement towards POWER over the past few years, specifically for databases, This is the natural time to simplify architectures and re-consolidate those Oracle Databases back onto the SPARC Network Management platforms, again. Why introduce the complexities or firewalls, multiple architectures, multiple code bases, multiple reboot windows, multiple maintenance windows, and overcomplicating D-R procedures when it is cheaper to put it all back on a new low end SPARC platform, and it can be made even less expensive by introducing virtualization technologies like [Oracle VM for SPARC] LDoms and [CPU Capped] Zones?

Monday, December 5, 2011

Small Solaris


Small Solaris

Abstract:
Solaris has traditionally been an operating system designed to run in a small footprint. Early Sun workstations, like the Sun 3/50 required only 4 Megabytes of RAM. The growth of memory requirements continued with the release of each operating system. Solaris 10 was the last commercial Solaris release to support 128Meg of RAM, which Solaris 10 Update 1 reportedly required 384Meg of RAM minimum. This size continues to grow, with added functionality.


EON:
There is a release of OpenSolaris referred to as EON or Embedded Operating System / Network. It is small enough to run from a 256Meg flash, but it should be run on a system with 1 Gig of RAM minimum.

Illumos Discussion:
An Illumos discussion thread yielded Jerry Kemp, who happened to reference a particular defunct appliance-discussion list where OpenSolaris was running on a Soekris net5501 system. Another system was mentioned to host OpenSolaris, the fit-PC. Jerry also mentioned two different blogs postings from Sun/Oracle's Jim Conners and also a note about Compressed ZFS on ARM port of OpenSolaris.

Physically Small:
Jim built one embedded system was a very small platform, but it included 512Meg of RAM. This is hardly small, by any sense of the meaning, but it is physically small!
[2008-11-08] - Physically small platform, 512Meg RAM

Small Footprint Framework:
Jim built a framework which creates a Solaris in-memory installation that will work in an area as small as 60 Megabytes.
[2007-02-07] - Framework to Help Create Small Footprint RAM Resident Solaris


ARM and CZFS:
In 2009, Vineeth Pillai from Sun Microsystems in the Czech Republic presented "OpenSolaris ARM Port and Its Future".

The OpenSolaris port to ARM was announced in June 8, 2009.
It was based on OpenSolaris 2008.05 build 86 and ported to NEC NaviEngine 1. Compressed ZFS is incompatible with ZFS, but more suited for embedded devices.

On June 19, 2009 - it was confirmed that UFS and ZFS were in the ARM port of OpenSolaris.

In June 25, 2009 - NEC contributed ARM code to make ZFS use 8 Megabytes of RAM to run ZFS and 4 Megabytes of RAM for ARC. By Compressing ZFS data structures, they managed to boot OpenSolaris in 16MB of RAM and bring the ARC to 1MB with a 2MB ZFS runtime!

Darren Moffat posed a question in September 28, 2009 about mounting a disk under a QEMU instance of ARM OpenSolaris where Mitsuru Sasanuma replied the NE1 emulator does not implement IDE (SATA) and NIC devices, so hard disk images could not be used in QEMU. CZFS could be used with regular files.

Illumos Implications:
The substantial question of the week is, can we move Illumos to something smaller or bring Illumos to embedded devices for USB external hard drives?

Friday, December 2, 2011

X Tab: OpenWindows Augmented Compatibility Environment



The following has been added to the X Tab for Solaris 9 and Solaris 10.
OpenWindows Augmented Compatibility Environment

owacomp - [http|ix86|sparc|src|readme] - OpenWindows acomp Project
olvwm4.4p4 - [http|pkg|src|readme] - Solaris 8 SPARC OpenLook Virtual Window Manager
olvwm4.4p4 - [http|pkg|src|readme] - Solaris 8 ix86 OpenLook Virtual Window Manager

Thursday, December 1, 2011

Oracle Database Appliance Webcast



Oracle Database Appliance Webcast

Don't miss the webcast on December 13, 2011 at 12:00EST noon!

Objectives to achieve from webcast attendance includes understanding:



  • Consolidation of many small databases into a single highly available solution

  • Deploy and Manage clustered systems in hours

  • Benefit from Single Vendor support for Hardware, OS, and Database
The featured speakers scheduled are:



  • Bob Thome
    Senior Director of Product Management,
    Oracle


  • Matthew Baier
    Director of Product Marketing,
    Oracle
Register Now - See You There!

Friday, November 18, 2011

Solaris 11: Updated Archive and Linking

Solaris 11: Updated Archive and Linking

Abstract:
Binaries in operating systems need to be identified so the underlying kernel knows how to execute them. Traditionally, binaries were much smaller than data files, so the transition to 64bit operating systems and chips never needed to address huge binaries. A 40 year old 32bit standard, created under AT&T UNIX SVR4, was getting "long in the tooth" and showing it's age. This was finally addressed with with Solaris 11 release.

The History:
An exhaustive document to the Solaris Linker and Library guide is published by Oracle. Solaris was built upon the System V Release 4 Linker Mapfile Format. A short article on the Linux linker from 2002 may be interesting to some. Discussion started in 2009 about enhancing Solaris, while retaining backwards compatibility.

[2009-12-22] PSARC/2009/688 - human readable and extensible ld mapfile syntax
[2010-01-06] Developer describes Solaris SVR4 32 bit Link Editor Map File Issues
[2010-01-07] Developer describes new Solaris Map File Syntax
[2011-11-11] The need for binaries exceeding 32bit lengths
[2011-11-11] Developer publishes modern archive/linking feature

You can reference the published document from Oracle for a complete listing of Solaris linking features, from Solaris 8 leading up to Solaris 11 Express.

Thursday, November 17, 2011

Solaris and Virtualization: KVM and Xen

[Solaris Logo, courtesy former Sun Microsystems]

Solaris and Virtualization: KVM and Xen
Abstract:
Solaris Intel looked to be a worthy operating system for the Intel marketspace with key technolgy such as ZFS, DTrace, Containers, and Xen. With Oracle's killing of Xen for Solaris 11 Intel, many were concerned that the benefits of Solaris on a server would be lost, without platform virtualization such as Xen. The Open Source Solaris community decided to invest in another technology, bringing true managed services scalability to the internet cloud environment.

[kvm logo, courtesy linux-kvm.org]
What is KVM:
KVM is the traditional acronym for "Keyboard, Video, Mouse" - as in KVM Switch. Unfortunately, some not-so-genius decided to overload the multi-decade old acronym with a new moniker - "Kernel Virtual Machine." The QEMU emulator is used as the foundation for the virtual machine. KVM was implemented to be tightly tied to the Linux kernel, with little attempt to be open and friendly to other system kernels, as was Xen.

[Joyent logo, courtesy joyent.com]
Cloud Computing: Joyent
Joyent moved KVM to the cloud with Solaris on their managed services grade SmartOS - encapsulating the finest of today's OS technologies: Zones, ZFS, DTrace, Crossbow. The use of KVM and QEMU within a Solaris Zone securely protects the hypervisor from virtual machine hacking exploits, which would ordinarily result in a disaster under Linux. The use of Crossbow provide for securiy and capping of network resources, through the underlying Solaris OS Kernel, to contain network exploits in the guest operating systems and/or their applicaions. Solaris DTrace provides unprecendeted KVM and guest operating system visibility - which includes cloud analytics from the hypervisor, operating system, middleware, and all the way down to the application layers.

Experiences porting KVM to SmartOS from bcantrill

The porting of  KVM to be encapsulated in a Solaris Zone within SmartOS was genius.

[Xen Logo, courtesy xenproject.org]

Conclusion:
The death of Xen under Solaris at the hands of Oracle was an untimely demise for a great concept - robust cloud computing could have been owned by Solaris, making others vendors look like a cheap knock-offs. While Oracle may have marketed Solaris 11 as the operating system of cloud computing, the real innovation came from Joyent, who implemented a multi-vendor cloud on top of managed services grade OpenSolaris fork, through the port of highly proprietary (operating system specific) KVM. Kudos to SmartOS team! Will HP, IBM, or Dell purchase Joyent for their SmartOS?

Wednesday, November 16, 2011

Enabling VNC in Solaris Express 11


Enabling VNC Under Solaris 11 Express

Abstract:
Open Systems have traditionally been accessed via Command Line. MIT create a fully object oriented, multi-tiered, open source windowing system called X Windows, which was quickly adopted by nearly all computing industry players. While X Windows is well suited for local area network technology, the need for wide area network technology was addressed through several different attempts, such as X11R6 "Broadway" and proxies leveraging compression. A lighter WAN suitable screen display protocol, referred to as Virtual Network Computing (VNC) is also commonly used for X displays.

Procedure:
Solaris 10 was shipped with VNC, instructions for enabling VNC under Solaris are found here.

Solaris 11 Express was shipped with a basic VNC service mostly available. The procedure to fully enable VNC under Solaris 11 is as follows:
  • Load Gnome Desktop Manager
# pkg install slim_install
  • After the ~400mb package is installed note the following issue:
# svcs consolekit
STATE STIME FMRI maintenance 1999 svc:/system/consolekit:default
The GDM depends on the console-kit-daemon for "defining and tracking users"
(console-kit-daemon man page). A generic error will occur telling you that a consolekit dependency is restarting too quickly.
  • Check the consolekit dependencies for restarting:
# svcs -d consolekit
onl
ine 1999 svc:/system/dbus:default
online 1999 svc:/system/filesystem/local:default
The -r is to recursively enable the daemon and then dependencies in the correct order.
#svcadm disable dbus
#svcadm clear consolekit

#svcadm enable -r consolekit
# vi /etc/gdm/custom.conf
[xdmcp] Enable=true
wq
  • A restart of the Gnome Display Manager and enabling of Xvnc are the last steps.
# svcadm restart gdm
# svccfg -s x11-server setprop options/tcp_listen=true
# svcadm enable xvnc-inetd

Tuesday, November 15, 2011

Japan and Architecture Independence


Japan and Architecture Independence

Abstract:
For a period of time, there was competition in the area of super computers with varied architectures, with standards based processors like SPARC leading other vendors. Most of the worlds modern super computers have been based upon American Intel based MPP platforms. Intel compatible CPU maker AMD started producing CPU's for supercomputers during recent years. There has also been an aggressive trend to move towards using graphical co-processor units. Most recently, nations have been trying to develop their own intellectual property.

Japan and CPU Architectures:
Japan has been moving towards computing independence by investing in their own implementation of SPARC processor.
[2011-06-20] Japan's Fujitsu creates fastest supercomputer at 8 Petaflops with SPARC64
[2011-11-02] Japan's Fujitsu tops its own fastest supercomputer with 10 Petaflop SPARC64
[2011-11-07] Japan's Fujitsu announces commercial supercomputer with 23 Petaflops SPARC64
[2011-11-21] Japan's Fujitsu 16 core SPARC64-IXfx Details Revealed *updated 11-22*

Conclusion:
It has long been concluded the Intel owned the Super Computer market, before vendors with GPU started to compete. With Japan's Fujitsu corporation, it is clear that the market is no longer going to be dominated by proprietary Intel architecture, but companies developing Open Architectures like SPARC will begin to compete again, as they did in the 1990's.

Monday, November 14, 2011

Solaris Tab: Solaris 11 Reference Additions

Solaris Tab: Solaris 11 Reference Additions

The following has been added to the Solaris Tab for Solaris 11 Reference information.

Solaris Reference Material
2011-11 [HTML] Solaris 11 Library
2011-11 [HTML] Solaris 11 Administration: Zones and Resource Management
2011-11 [HTML] Solaris 11 Administration: ZFS File Systems
2011-11 [HTML] Solaris 11 Administration: Adding and Updating Solaris Packages
2011-11 [HTML] Solaris 11 Administration: Creating and Managing Boot Environments
2011-11 [PDF] What's New in Solaris 11 11/11
2011-11 [PDF] Solaris 11 Cheat Sheet for Image Packaging System

Friday, November 11, 2011

Happy Last Binary Day!!!

Sadly, this will be the last binary day, in modern history.

01/01/01
01/10/01
01/11/01
10/01/01
10/10/01
10/11/01
11/01/01
11/10/01
11/11/01

01/01/10
01/10/10
01/11/10
10/01/10
10/10/10
10/11/10
11/01/10
11/10/10
11/11/10

01/01/11
01/10/11
01/11/11
10/01/11
10/10/11
10/11/11
11/01/11
11/10/11
11/11/11

Happy Last Binary Day!

Solaris 11: Oracle Launches Cloud OS


Solaris: Oracle Launches Cloud OS

Launch Video:
Oracle launched Solaris 11 on November 11, 2010, with real-time streaming. The replay is available.

Highlights:

Oracle President Mark Hurd opens the Solaris 11 Launch

  • Connected Cloud Management
  • Zero Overhead Virtualization
    This is Zones, formerly Solaris Containers.
  • Cloud Scale
    True linear scalability
  • 2700 Projects
  • 400 innovations
  • 750 customers using Solaris 11

Mark Hurd also talks about how Solaris is the #1 UNIX:
  • #1 for UNIX shipments for past decade
  • #1 for Oracle deployments
  • Largest application portfolio breaking 11,000+
  • Over 60,000 Solaris Customers
Investment protection is discussed: upgradable from T4 to T5. SPARC SuperCluster solution brings together benefits of SPARC with benefits of Exadata.

Oracle showed a quick video with various testimonials:
  • Solaris customers are aggressive users of technology
    Use technology to differentiate their businesses from others
  • Decrease total cost of ownership (TCO)
    Provide high performance and highly scalable engineered solutions
  • Solaris scales from a single node to multiple nodes
  • Multi-threaded architecture
    Run more applications and process more transactions with fewer CPU's
  • ZFS is the core capability to drive use of Solaris 11
    Manages large amounts of data, mirrors across chassis
  • Oracle Non-Global Zones and Oracle RAC
    74% reduction in support costs
  • Compare Oracle SPARC, HP HPUX, and IBM AIX
    Competition costs 3x more than SPARC Solaris
  • Scalability & Securely support dramatic growth

Oracle Executive Vice President John Fowler takes the stage

Talks about Solaris 11
  • #1 UNIX, incorporate into cloud deployments, engineer to run Oracle better
  • Solaris has more deployments than AIX and HPUX combined
  • 11,000 certified independent software vendor applications for Solaris
  • Thousands of companies produce software for Solaris
  • Cloud level deployment for Solaris
  • Operating Systems get better over time as they add capability and features
The opening statement:
  • Optimized to run all of your applications faster
    Solaris runs Everything
    - the most critical applications
    - tier 2 or tier 3 applications
The major points:


  1. Kernel Enhancements - Built for next-decade hardware
    - hundreds of cores, thousands of threads
    - tens of terabytes of memory
    - double digit gigabyte network performance
    - hundreds of improvements in Solaris
    - engineering through the stack provides best performance
    - best performance in all application tiers
    - engineered for scale
    - engineered for high performance applications
    - engineered for efficiency in running the oracle stack and other applications

  1. Virtualization - Designed in virtualization at every layer
    - don't build toys, we are not a toy company
    - supported, high-service, high-availability, secure mission critical applications
    - availability designed at every level, not just a virtual machine

  2. Data - at a cloud
    - SAN, NAS, share block devices, share file systems
    - common data services in the operating system
    - de-duplication, compression, encryption, flash aware storage pools
    - move storage services next to application



  3. Security - Engineered Security at Every Layer of System
    - Defense in depth
    - Multi-tenancy in design
    - delegated administration
    - robust auditing
    - immutable zones
    - network & data layer protection
    - encrypted data per-tenant



  4. Life Cycle Management - Cloud Deployment
    - Cloud repository for packages and patches
    - Local repositories, key validated & encrypted
    - patch updates in the orders of minutes instead of hours
    - fast reboots of zones
    - cloned zones with rollback options
    - proven Oracle VM templates for Oracle run time applications
    - incorporate own software applications into repositories
    - rapid and safe deployment and rollback
Steve Wilson of Vice President of Engineering, Systems
  1. Management Story - Hardware, OS, and Application
    - Solaris 11 host self-service cloud environment
    - Connect to cloud services at oracle
    - Ability to manage Oracle applications
  2. Solaris 11 Enterprise Manager Ops Center
    - Dashboard for self-service cloud on Solaris 11

    - Visibility of clouds, storage, servers, switches, networks, disks, volumes

    - Visibility of users, quotas, usage

    - Visibility to capacity issues, impending failures, security patches

    - Automatic service request creation at Oracle
    - Ability to apply patches to thousands of servers at once, according to policy
    - OS Level Analytics
  3. Oracle Enterprise Cloud Control Integration
    - Ops Center Drill-Down

    - Review Analytics correlated to Oracle Application issue

    - Update operating system characteristics (i.e. network flow)

Oracle Executive Vice President John Fowler takes the stage for Solaris SPARC updates
  • SPARC and Solaris 5 Year Roadmap
    - nearly half-complete
    - all on-time
    - some future products in labs today
    - will continue the drum beat of products
  • Binary Compatibility
  • Investment Protection
  • No forced migration from Solaris 10 to Solaris 11
  • Constant output of activity
  • Classic Microprocessor Investment in SPARC
  • Adding Enterprise Application Acceleration
    - De-Compression added for Oracle RDBMS
    - Acceleration for RAC clustering
    - Native support for underlying data formats
  • Solaris 12 will eventually come
Mark Hurd & John Fowler - Questions and Answers session
  • Mark - Strategy for supporting Solaris 11 on SPARC and x86
    Will support both!
    Solaris moved to x86 to embrace blade market, other UNIX platforms did not
  • John - Will new updates have to wait until Solaris 12?
    No, we provide stability for the customer, new capabilities will arrive in Solaris 11 Update 1
  • Mark - What is the most important reason to move from IBM to Oracle?
    Speed, Performance, TCO, Supportability, ISV availability, x86 availability, scalability, cloud connectability, steady stream of announcements, performance telemetry, commitment to product set
  • John - How long will Oracle support Solaris 10
    Solaris 1o update coming for T5 & M4, then a 10 year extended support
  • John - Oracle [RDBMS] is moving to 12c, when will Solaris be called 12c?
    Larry runs naming. Oracle RDBMS 12c will be rock-solid on Solaris
  • John - How secure is data in the cloud?
    Most people are not encrypting data in the cloud.
    Hardware does encryption at line-speed and wire-speed on the file system, network, and even in Oracle table space with virtually no impact. Oracle offers what no other provider does.
  • John - De-duplication is a huge improvement potential. How huge?
    This is huge, copies are made for data protection and archive, with huge numbers of copies. ZFS at the OS offers dedup and compression. Only OS which includes dedup built in.
  • John - Many customers are running Java, what makes Solaris good to run Java?
    Significant effort into: scheduler, memory manager, networking stack, compiler, observability into JVM, analytic tools
  • John - Policy question: solaris images on Amazon Elastic Cloud?
    Available on OTN for customers, today. Only an hour into launch!
  • John - Biggest question today.
    Life Cycle Management - Cloud accelerates this requirement. Cloud amplifies it, takes away SLA.
    Security - substantial technology

Thursday, November 10, 2011

China and Architecture Independence


China and Architecture Independence

Abstract:
For a period of time, there was competition in the area of super computers with varied architectures, with standards based processors like SPARC leading other vendors. Most of the worlds modern super computers have been based upon American Intel based MPP platforms. Intel compatible CPU maker AMD started producing CPU's for supercomputers during recent years. There has also been an aggressive trend to move towards using graphical co-processor units. Most recently, nations have been trying to develop their own intellectual property.

China and CPU Architectures:
China has been moving towards computing independence by investing in SPARC, MIPS, and ALPHA based processors.
[2008-02-26] - China Ministry of Education joins SPARC designers
[2010-01-10] - China designs MIPS based supercomputer
[2010-11-05] - China builds supercomputer with SPARC service node processor
[2011-10-31] - China designs ALPHA based supercomputer (video)

Conclusion:
It is not a good plan for an aspiring superpower to base their entire nations security on a single computing architecture, never mind a computer architecture which is foreign... or even worse, computing components manufactured exclusively by foreign nations.

Wednesday, November 9, 2011

Solaris Zones: Rapid Application Deployment


Solaris Zones: Rapid Application Development

Abstract:
Many development systems are based upon dedicated hardware infrastructure, which limits the number of simultaneous development releases. With Solaris Zones, multiple application releases can be developed at the same time on a platform, where the developed application is encapsulated, to be moved to test, and rapidly cloned into production.

Description:
Solaris Zones offer the following features:
  • isolation
  • rapid boot
  • built-in rollback
  • system mobility
  • cloning
Presentation
A use-case by a telephone carrier is illustrated in the following Solaris Zone presentation.
[html] Solaris Zones Dev, Test, and Deployment Presenation

Additional Information
[html] Solaris Zones, Containers, Resource Management Guide
[html] Solaris Zones Developer Guide

Tuesday, November 8, 2011

Apple: Breaking Free from Google?


Abstract:
During a time of peace, Google sat on the board of directors of Apple, and there was innovation, profit, and partnership in the minds of both companies. Apple engaged Google to provide network application services such as mapping, then Google betrayed Apple with this inside information. Apple has been trying to break free from Google, ever since.


History:

Apple started conceptual develpment of the iPhone, but Apple had a few gaps. Google sat on Apple's board of directors, so they made a good partner, to inquire about how they could fill some gaps. An agreement was struck, Google would provide mapping, instead of Apple going to other third-party mapping providers like Yahoo, who would not compete with them.

Google used this private information and purchased a development company who owned Android operating system. Google created a parallel development track, for their own Android based smart phone, without Apple's knowledge - the CEO of Google, who sat on Apple's board of directory was engaged in corporate espionage.

Apple released the ground-breaking iPhone, while Google put the finishing touches on their own smart phone. 6 months later, Google released their own smart phone, containing the same features, with a simlar look and feel.

Apple was stuck with having to return back to Google, as their mapping partner, Apple funded Google's mobile mapping through the iPhone revenue, while Google could take Apple's investment to compete with them into the mobile phone market. Google had clearly broken Apple's trust by using inside information from their CEO's participation in the Board of Directors, to gain an unfair competitive advantage to the rest of the market.


Breaking Free:

Google provided Apple with key functionality: mapping.

Unfortunately, mapping is a lot more complex than one might think. Roads are built & rebuilt. Continents shift. Earthquakes happen. Houses are demolished and turned into shopping malls. Farmland is turned into office parks. Progress occurs daily on a world-wide basis.

Apple's movement towards mapping independence started, not long after Google betrayed their business parter:
[2009-10-01] - Apple buys GigaOM "placebase", to gain location information
[2010-07-14] - Apple buys French-Canadian mapping Poly9, to have a world map
[2011-10-31] - Apple buys Sweedish 9to5Mac, to have a 3D world map


Freedom at Last?

The question in everyone's mind should be: when will Apple finally be able to claim freedom from Google, the company who double-crossed them, placing a bullet in Apple's leg, and crippling what could have allowed Apple's innovation to compete fairly in the Smart Phone industry?

Other companies, like Nokia and Symbian, had no chance to compete fairly with Google, who had left them with no way forward. Seeking partnership with Google, who so clearly betrayed Apple, would be likened to "making a deal with the devil" - phone vendors would know that Google would likely later cripple them in a similar fashion (Google stealing intellectual property while holding them prisoner with their own.)

One can only guess when Apple might break free. It might not be, until iPhone IOS 6, since it might take Apple a year to consolidate their 3 purchases over the past 3 years - unless Apple has another acquisition in mind, to surpass the capabilities of Google - the traitor in their midst.

Epilogue

Apple, Nokia, and Symbian were not the only companies crippled by Google's corporate espionage. Sun Microsystems had cooperated with Google with their central processor independant Java platform, with the understanding that Sun would receive license fees for Google's usage of Java in the mobile phones.

In the end, Sun received no license fees, as they did from other mobile vendors (like Motorola.) This placed mobile phone who licensed Java at a pricing disadvantage in the mobile phone arena, with those vendors who used Google's technology... thus encouraging those vendors to switch to Google's technology. Sun Microsystems continued to bleed money, until they were purchased by Oracle. Oracle filed law suits against Google.

Apple filed files law suits against third-party phone vendors, who used Android, in other knock-off Apple products. Google purchased Motorola (mobile phone company), undercutting Oracle's future revenue stream and gaining patent protection for their double-dealing. Sony broke their smart phone partnership with Symbian, who basically created the smartphone market.

When wondering about corporate morality, it is pretty clear - Google did evil...

Monday, November 7, 2011

Solaris Community Changes: genunix



Solaris Community Changes: genunix

Abstract:
When Sun started the process of opening Solaris, they created OpenSolaris, a port of Solaris based upon an open source code base. With the purchase of Sun by Oracle, for all intensive purposes, it appeared that OpenSolaris was dead, but there is still activity in the individual ports as well as the GenUNIX Mercurial Repositories.

Details:
While there may not seem to be a lot of activity in the Solaris Community on genunix.org, the Mercurial Repository shows a different pattern.

While the OS/Net repositories for OpenSolaris seem basically dead, for over a year, the pkg-gate and solaris userland consolidations are still incurring changes.

GenUNIX Mercurial Repositories:
The active respositories, as of October 2011 are as follows

Name Description Contact Last change
pkg-gate pkg(5) (aka IPS) repository Cyril Plisko 90 minutes ago
userland.hg Solaris Userland consolidation Cyril Plisko 2 weeks ago
g11n.hg Globalization (G11N) repository Cyril Plisko 3 weeks ago
sfwnv.hg Solaris Freeware repository Cyril Plisko 3 months ago

Of course, there are other places for active Solaris Open Communities to congregate (i.e. illumos), but at least people should be aware of where that there are a lot of changes still happening on genunix.org, even if the home page looks rather stale.

The community is waiting to see what happens, once Solaris 11 is released.

Friday, November 4, 2011

Oracle White Paper: Solaris 10 to 11 Changes


Oracle White Paper: Solaris 10 to 11 Changes

Abstract:
When an operating system is upgraded, there are various new features added and old features removed. With the release of Oracle 11 on November 2011, independant software vendors (ISV's) must be aware of the pending changes.

Details:
Oracle released an ISV Adpotion Guide for vendors moving to Solaris 11. This guide has been updated regularly by Oracle, as new features were added into Solaris 11 Express. It should be considered an authoritative reference for conducting software compatibility testing.

[pdf] September 2011 - Oracle Solaris 11 Express ISV Adoption Guide