Solaris 11: Updated Archive and Linking

Solaris 11: Updated Archive and Linking

Abstract:
Binaries in operating systems need to be identified so the underlying kernel knows how to execute them. Traditionally, binaries were much smaller than data files, so the transition to 64bit operating systems and chips never needed to address huge binaries. A 40 year old 32bit standard, created under AT&T UNIX SVR4, was getting "long in the tooth" and showing it's age. This was finally addressed with with Solaris 11 release.

The History:
An exhaustive document to the Solaris Linker and Library guide is published by Oracle. Solaris was built upon the System V Release 4 Linker Mapfile Format. A short article on the Linux linker from 2002 may be interesting to some. Discussion started in 2009 about enhancing Solaris, while retaining backwards compatibility.

[2009-12-22] PSARC/2009/688 - human readable and extensible ld mapfile syntax
[2010-01-06] Developer describes Solaris SVR4 32 bit Link Editor Map File Issues
[2010-01-07] Developer describes new Solaris Map File Syntax
[2011-11-11] The need for binaries exceeding 32bit lengths
[2011-11-11] Developer publishes modern archive/linking feature

You can reference the published document from Oracle for a complete listing of Solaris linking features, from Solaris 8 leading up to Solaris 11 Express.

Solaris and Virtualization: KVM and Xen

[Solaris Logo, courtesy former Sun Microsystems]

Solaris and Virtualization: KVM and Xen

Abstract:
Solaris Intel looked to be a worthy operating system for the Intel marketspace with key technolgy such as ZFS, DTrace, Containers, and Xen. With Oracle's killing of Xen for Solaris 11 Intel, many were concerned that the benefits of Solaris on a server would be lost, without platform virtualization such as Xen. The Open Source Solaris community decided to invest in another technology, bringing true managed services scalability to the internet cloud environment.

[kvm logo, courtesy linux-kvm.org]

What is KVM:
KVM is the traditional acronym for "Keyboard, Video, Mouse" - as in KVM Switch. Unfortunately, some not-so-genius decided to overload the multi-decade old acronym with a new moniker - "Kernel Virtual Machine." The QEMU emulator is used as the foundation for the virtual machine. KVM was implemented to be tightly tied to the Linux kernel, with little attempt to be open and friendly to other system kernels, as was Xen.

[Joyent logo, courtesy joyent.com]

Cloud Computing: Joyent
Joyent moved KVM to the cloud with Solaris on their managed services grade SmartOS - encapsulating the finest of today's OS technologies: Zones, ZFS, DTrace, Crossbow. The use of KVM and QEMU within a Solaris Zone securely protects the hypervisor from virtual machine hacking exploits, which would ordinarily result in a disaster under Linux. The use of Crossbow provide for securiy and capping of network resources, through the underlying Solaris OS Kernel, to contain network exploits in the guest operating systems and/or their applicaions. Solaris DTrace provides unprecendeted KVM and guest operating system visibility - which includes cloud analytics from the hypervisor, operating system, middleware, and all the way down to the application layers.

Experiences porting KVM to SmartOS from bcantrill

The porting of  KVM to be encapsulated in a Solaris Zone within SmartOS was genius.

[Xen Logo, courtesy xenproject.org]

Conclusion:

The death of Xen under Solaris at the hands of Oracle was an untimely demise for a great concept - robust cloud computing could have been owned by Solaris, making others vendors look like a cheap knock-offs. While Oracle may have marketed Solaris 11 as the operating system of cloud computing, the real innovation came from Joyent, who implemented a multi-vendor cloud on top of managed services grade OpenSolaris fork, through the port of highly proprietary (operating system specific) KVM. Kudos to SmartOS team! Will HP, IBM, or Dell purchase Joyent for their SmartOS?

Enabling VNC in Solaris Express 11

Enabling VNC Under Solaris 11 Express

Abstract:
Open Systems have traditionally been accessed via Command Line. MIT create a fully object oriented, multi-tiered, open source windowing system called X Windows, which was quickly adopted by nearly all computing industry players. While X Windows is well suited for local area network technology, the need for wide area network technology was addressed through several different attempts, such as X11R6 "Broadway" and proxies leveraging compression. A lighter WAN suitable screen display protocol, referred to as Virtual Network Computing (VNC) is also commonly used for X displays.

Procedure:
Solaris 10 was shipped with VNC, instructions for enabling VNC under Solaris are found here.

Solaris 11 Express was shipped with a basic VNC service mostly available. The procedure to fully enable VNC under Solaris 11 is as follows:

• Load Gnome Desktop Manager

# pkg install slim_install

• After the ~400mb package is installed note the following issue:

# svcs consolekit
STATE STIME FMRI maintenance 1999 svc:/system/consolekit:default

The GDM depends on the console-kit-daemon for "defining and tracking users"
(console-kit-daemon man page). A generic error will occur telling you that a consolekit dependency is restarting too quickly.

• Check the consolekit dependencies for restarting:

# svcs -d consolekit
online 1999 svc:/system/dbus:default
online 1999 svc:/system/filesystem/local:default

The -r is to recursively enable the daemon and then dependencies in the correct order.

#svcadm disable dbus
#svcadm clear consolekit
#svcadm enable -r consolekit

• A console will need to be provided to the passive VNC X Server through X Display Management Control Protocol.

# vi /etc/gdm/custom.conf
[xdmcp] Enable=true
wq

• A restart of the Gnome Display Manager and enabling of Xvnc are the last steps.

# svcadm restart gdm
# svccfg -s x11-server setprop options/tcp_listen=true
# svcadm enable xvnc-inetd

Japan and Architecture Independence

Japan and Architecture Independence

Abstract:
For a period of time, there was competition in the area of super computers with varied architectures, with standards based processors like SPARC leading other vendors. Most of the worlds modern super computers have been based upon American Intel based MPP platforms. Intel compatible CPU maker AMD started producing CPU's for supercomputers during recent years. There has also been an aggressive trend to move towards using graphical co-processor units. Most recently, nations have been trying to develop their own intellectual property.

Japan and CPU Architectures:
Japan has been moving towards computing independence by investing in their own implementation of SPARC processor.
[2011-06-20] Japan's Fujitsu creates fastest supercomputer at 8 Petaflops with SPARC64
[2011-11-02] Japan's Fujitsu tops its own fastest supercomputer with 10 Petaflop SPARC64
[2011-11-07] Japan's Fujitsu announces commercial supercomputer with 23 Petaflops SPARC64
[2011-11-21] Japan's Fujitsu 16 core SPARC64-IXfx Details Revealed *updated 11-22*

Conclusion:
It has long been concluded the Intel owned the Super Computer market, before vendors with GPU started to compete. With Japan's Fujitsu corporation, it is clear that the market is no longer going to be dominated by proprietary Intel architecture, but companies developing Open Architectures like SPARC will begin to compete again, as they did in the 1990's.

Solaris Tab: Solaris 11 Reference Additions

Solaris Tab: Solaris 11 Reference Additions

The following has been added to the Solaris Tab for Solaris 11 Reference information.

Solaris Reference Material
2011-11 [HTML] Solaris 11 Library
2011-11 [HTML] Solaris 11 Administration: Zones and Resource Management
2011-11 [HTML] Solaris 11 Administration: ZFS File Systems
2011-11 [HTML] Solaris 11 Administration: Adding and Updating Solaris Packages
2011-11 [HTML] Solaris 11 Administration: Creating and Managing Boot Environments
2011-11 [PDF] What's New in Solaris 11 11/11
2011-11 [PDF] Solaris 11 Cheat Sheet for Image Packaging System

Happy Last Binary Day!!!

Sadly, this will be the last binary day, in modern history.

01/01/01
01/10/01
01/11/01
10/01/01
10/10/01
10/11/01
11/01/01
11/10/01
11/11/01

01/01/10
01/10/10
01/11/10
10/01/10
10/10/10
10/11/10
11/01/10
11/10/10
11/11/10

01/01/11
01/10/11
01/11/11
10/01/11
10/10/11
10/11/11
11/01/11
11/10/11
11/11/11

Happy Last Binary Day!

Solaris 11: Oracle Launches Cloud OS

Solaris: Oracle Launches Cloud OS

Launch Video:
Oracle launched Solaris 11 on November 11, 2010, with real-time streaming. The replay is available.

Highlights:

Oracle President Mark Hurd opens the Solaris 11 Launch

• Connected Cloud Management
• Zero Overhead Virtualization
  This is Zones, formerly Solaris Containers.
• Cloud Scale
  True linear scalability
• 2700 Projects
• 400 innovations
• 750 customers using Solaris 11

Mark Hurd also talks about how Solaris is the #1 UNIX:

• #1 for UNIX shipments for past decade
• #1 for Oracle deployments
• Largest application portfolio breaking 11,000+
• Over 60,000 Solaris Customers

Investment protection is discussed: upgradable from T4 to T5. SPARC SuperCluster solution brings together benefits of SPARC with benefits of Exadata.

Oracle showed a quick video with various testimonials:

• Solaris customers are aggressive users of technology
  Use technology to differentiate their businesses from others
• Decrease total cost of ownership (TCO)
  Provide high performance and highly scalable engineered solutions
• Solaris scales from a single node to multiple nodes
• Multi-threaded architecture
  Run more applications and process more transactions with fewer CPU's
• ZFS is the core capability to drive use of Solaris 11
  Manages large amounts of data, mirrors across chassis
• Oracle Non-Global Zones and Oracle RAC
  74% reduction in support costs
• Compare Oracle SPARC, HP HPUX, and IBM AIX
  Competition costs 3x more than SPARC Solaris
• Scalability & Securely support dramatic growth

Oracle Executive Vice President John Fowler takes the stage

Talks about Solaris 11

• #1 UNIX, incorporate into cloud deployments, engineer to run Oracle better
• Solaris has more deployments than AIX and HPUX combined
• 11,000 certified independent software vendor applications for Solaris
• Thousands of companies produce software for Solaris
• Cloud level deployment for Solaris
• Operating Systems get better over time as they add capability and features

The opening statement:

• Optimized to run all of your applications faster
  Solaris runs Everything
  - the most critical applications
  - tier 2 or tier 3 applications

The major points:

1. Kernel Enhancements - Built for next-decade hardware
   - hundreds of cores, thousands of threads
   - tens of terabytes of memory
   - double digit gigabyte network performance
   - hundreds of improvements in Solaris
   - engineering through the stack provides best performance
   - best performance in all application tiers
   - engineered for scale
   - engineered for high performance applications
   - engineered for efficiency in running the oracle stack and other applications

1. Virtualization - Designed in virtualization at every layer
   - don't build toys, we are not a toy company
   - supported, high-service, high-availability, secure mission critical applications
   - availability designed at every level, not just a virtual machine
   
   
2. Data - at a cloud
   - SAN, NAS, share block devices, share file systems
   - common data services in the operating system
   - de-duplication, compression, encryption, flash aware storage pools
   - move storage services next to application




3. Security - Engineered Security at Every Layer of System
   - Defense in depth
   - Multi-tenancy in design
   - delegated administration
   - robust auditing
   - immutable zones
   - network & data layer protection
   - encrypted data per-tenant
   
   
   
   
4. Life Cycle Management - Cloud Deployment
   - Cloud repository for packages and patches
   - Local repositories, key validated & encrypted
   - patch updates in the orders of minutes instead of hours
   - fast reboots of zones
   - cloned zones with rollback options
   - proven Oracle VM templates for Oracle run time applications
   - incorporate own software applications into repositories
   - rapid and safe deployment and rollback

Steve Wilson of Vice President of Engineering, Systems

1. Management Story - Hardware, OS, and Application
   - Solaris 11 host self-service cloud environment
   - Connect to cloud services at oracle
   - Ability to manage Oracle applications
   
2. Solaris 11 Enterprise Manager Ops Center
   - Dashboard for self-service cloud on Solaris 11
   
   - Visibility of clouds, storage, servers, switches, networks, disks, volumes
   
   - Visibility of users, quotas, usage
   
   - Visibility to capacity issues, impending failures, security patches
   
   - Automatic service request creation at Oracle
   - Ability to apply patches to thousands of servers at once, according to policy
   - OS Level Analytics
   
3. Oracle Enterprise Cloud Control Integration
   - Ops Center Drill-Down
   
   - Review Analytics correlated to Oracle Application issue
   
   - Update operating system characteristics (i.e. network flow)

Oracle Executive Vice President John Fowler takes the stage for Solaris SPARC updates

• SPARC and Solaris 5 Year Roadmap
  - nearly half-complete
  - all on-time
  - some future products in labs today
  - will continue the drum beat of products
• Binary Compatibility
• Investment Protection
• No forced migration from Solaris 10 to Solaris 11
• Constant output of activity
• Classic Microprocessor Investment in SPARC
• Adding Enterprise Application Acceleration
  - De-Compression added for Oracle RDBMS
  - Acceleration for RAC clustering
  - Native support for underlying data formats
• Solaris 12 will eventually come

Mark Hurd & John Fowler - Questions and Answers session

• Mark - Strategy for supporting Solaris 11 on SPARC and x86
  Will support both!
  Solaris moved to x86 to embrace blade market, other UNIX platforms did not
• John - Will new updates have to wait until Solaris 12?
  No, we provide stability for the customer, new capabilities will arrive in Solaris 11 Update 1
• Mark - What is the most important reason to move from IBM to Oracle?
  Speed, Performance, TCO, Supportability, ISV availability, x86 availability, scalability, cloud connectability, steady stream of announcements, performance telemetry, commitment to product set
• John - How long will Oracle support Solaris 10
  Solaris 1o update coming for T5 & M4, then a 10 year extended support
• John - Oracle [RDBMS] is moving to 12c, when will Solaris be called 12c?
  Larry runs naming. Oracle RDBMS 12c will be rock-solid on Solaris
• John - How secure is data in the cloud?
  Most people are not encrypting data in the cloud.
  Hardware does encryption at line-speed and wire-speed on the file system, network, and even in Oracle table space with virtually no impact. Oracle offers what no other provider does.
• John - De-duplication is a huge improvement potential. How huge?
  This is huge, copies are made for data protection and archive, with huge numbers of copies. ZFS at the OS offers dedup and compression. Only OS which includes dedup built in.
• John - Many customers are running Java, what makes Solaris good to run Java?
  Significant effort into: scheduler, memory manager, networking stack, compiler, observability into JVM, analytic tools
• John - Policy question: solaris images on Amazon Elastic Cloud?
  Available on OTN for customers, today. Only an hour into launch!
• John - Biggest question today.
  Life Cycle Management - Cloud accelerates this requirement. Cloud amplifies it, takes away SLA.
  Security - substantial technology