Tuesday, February 8, 2011

Comparing Packages Between Platforms


Comparing Packages Beteen Plataforms

Abstract:
When working in a clustered environment, it is often a requirement to see if the appropriate packages have been installed on all platforms in the cluster. The number of packages on a platform are many, but a simple script can be helpful.

Packaging Technology:
The industry standard packaging for UNIX systems is UNIX SVR4 packaging. Standard tools for packaging include: pkgadd, pkginfo, pkgtrans, pkgrm, pkgmk, pkgchk, pkgparam, pkgproto, pkgadm.

A long utput from the pkginfo command follows:

sun9/user$ pkginfo -l HPNP                                             

PKGINST: HPNP
NAME: JetAdmin for Unix
CATEGORY: application
ARCH: sparc
VERSION: D.06.15
BASEDIR: /
VENDOR: HP
DESC: HP Network Printer support package
PSTAMP: odybld3981208144215
INSTDATE: Aug 12 2005 08:50
STATUS: completely installed
FILES: 348 installed pathnames
6 shared pathnames
32 directories
238 executables
13353 blocks used (approx)



For a cursory view of a system, the pkginfo command provides basic information required for cursory consistency checks.

Simple Check:
If the identical install media is used, a simple post-install check may be desired across multiple platforms in a cluster. An sample script follows where sun1, sun2, sun3, sun4 are located on a network where temporary directories are shared via NFS and automounting is enabled.

sun1/user$ pkginfo >/net/sun4/tmp/sun1.packages
sun2/user$ pkginfo >/net/sun4/tmp/sun2.packages
sun3/user$ pkginfo >/net/sun4/tmp/sun3.packages
sun4/user$ cd /tmp

sun4/user$ nawk ' BEGIN { Pattern="%35s%35s%35s%35s\n" }
FILENAME=="sun1.packages" { sun1[$2]=$2 ; Name[$2]=$2 }
FILENAME=="sun2.packages" { sun2[$2]=$2 ; Name[$2]=$2 }
FILENAME=="sun3.packages" { sun3[$2]=$2 ; Name[$2]=$2 }
END {
printf Pattern,"Common","sun1","sun2","sun3"
for ( i in Name ) printf Pattern,i,sun1[i],sun2[i],sun3[i]
}' *.packages sort nawk 'NF<4'
A simple output of 4 columns is produced, with differences.

   SMCdb       SMCdb     SMCdb                      
SMCtk SMCtk SMCtk
SFWatk SFWatk SFWatk
SMCgcc SMCgcc SMCgcc
SMCtcl SMCtcl SMCtcl
SMCxpm SMCxpm SMCxpm
SUNWdc SUNWdc SUNWdc
SUNWus SUNWus
TSIpgx TSIpgx
SFWdbus SFWdbus SFWdbus
SFWgtk2 SFWgtk2 SFWgtk2
SMCgdbm SMCgdbm SMCgdbm
SMCntop SMCntop SMCntop
SMCossl SMCossl SMCossl
SMCpcre SMCpcre SMCpcre
SMCrrdt SMCrrdt SMCrrdt
SMEvplr SMEvplr
SMEvplu SMEvplu
SUNWaac SUNWaac
SUNWafb SUNWafb
SUNWbdb SUNWbdb
...
Versioning and Integrity Checks:

In order to test for proper versions and package integrity, there are other commands which can be leveraged:
  • pkgchk
    Check detailed integrity of files associated with packages, including existence, permissions, etc.
  • pkginfo -l
    Check versioning, architecture, dates, install integrity, etc.
The package checking script can be enhanced with such scripts for more robust checking.

Tuesday, February 1, 2011

Primer: Solaris 10 Update 9



Primer: Solaris 10 Update 9

During the installation of Solaris 10 Update 9, there may be several odd symptoms that might catch an installer by surprise.

Sluggish or Long Login Time
Takes 20 seconds run quota during a login

If logging into a server with "rlogin", "telnet" or "ssh" taking a long time, try to perform a "prstat" during the login process. You may get about 20 seconds to figure out what is going wrongly. The "root" user may not experience the delay.

If "quota" is showing up for an extended period of time, it may be due to a search over NFS mounted file systems without NFS being configured correctly. This can be alleviated by unmounting the mounted NFS file system. The "mount | grep nfs" will help to identify the nfs file systems (you can normally ignore "vold".)
sunv890/user$ nawk '/nfs/ && !/vold/' /etc/mnttab
sunt2000:/u000/prodsupt /mnt nfs rw,xattr,dev=5ec0004 1296682012
sunv890/dh127087$ time quota
real 0m20.03s ...
sunv890/root$
umount /mnt
sunv890/user$
nawk '/nfs/ && !/vold/' /etc/mnttab
sunv890/user$
time quota
real 0m0.02s
...
Ensure quota and nfs partitions are properly configured on a newly installed system.

Occasional Network Failures
node name or service name not known

Some common operations may occasionally fail, for no apparent reason, such as: ping, telnet, ftp, etc. This is usually a name resolution issue. DNS is more commonly run by non-robust operating systems, so these types of errors may become more frequent.

If you have a variety of servers in a cluster or network management servers, where the ip addresses do not change - one may wish to basically guarantee the name resolution for those connections attempts by adding those ip & server entries to the "/etc/hosts" file and adjusting your "hosts:" option in the "/etc/nsswitch.conf" to perform a host table lookup before going to DNS via: "hosts: files dns".

v890/user$ grep host /etc/nsswitch.conf
# "hosts:" and "services:" in this file are used only if the
#hosts: dns files
hosts: files dns
Don't let an consumer appliances, immature or consumer grade operating systems place your mission critical operation at risk.

Service Management Facility
How to tell what services are available

Solaris 10 offers a large variety of services to the user and application community. If functionality you are expecting is not working "out of the box", it is a good chance there is a security reason for it. The best place to start is by getting a description of that services are available.

Ultra60/root# svcs -o FMRI,DESC | sort
FMRI DESC
lrc:/etc/rc2_d/S10lu -
lrc:/etc/rc2_d/S20sysetup -
lrc:/etc/rc2_d/S40llc2 -
lrc:/etc/rc2_d/S42ncakmod -
lrc:/etc/rc2_d/S47pppd -
lrc:/etc/rc2_d/S70uucp -
lrc:/etc/rc2_d/S72autoinstall -
lrc:/etc/rc2_d/S73cachefs_daemon -
lrc:/etc/rc2_d/S81dodatadm_udaplt -
lrc:/etc/rc2_d/S89bdconfig -
lrc:/etc/rc2_d/S89PRESERVE -
lrc:/etc/rc2_d/S90loc_ja_cssd -
lrc:/etc/rc2_d/S91ifbinit -
lrc:/etc/rc2_d/S91jfbinit -
lrc:/etc/rc2_d/S94ncalogd -
lrc:/etc/rc2_d/S98deallocate -
lrc:/etc/rc3_d/S16boot_server -
lrc:/etc/rc3_d/S50apache -
lrc:/etc/rc3_d/S52imq -
lrc:/etc/rc3_d/S80mipagent -
lrc:/etc/rc3_d/S84appserv -
lrc:/etc/rc3_d/S84patchserver -
svc:/application/cde-printinfo:default CDE Print Viewer
svc:/application/font/fc-cache:default FontConfig Cache Builder
svc:/application/font/stfsloader:default Standard Type Services Framework (STSF) Font Server loader
svc:/application/graphical-login/cde-login:default CDE login
svc:/application/management/dmi:default Sun Solstice Enterprise DMI
svc:/application/management/seaport:default net-snmp SNMP daemon
svc:/application/management/sma:default net-snmp SNMP daemon
svc:/application/management/snmpdx:default Sun Solstice Enterprise Master Agent
svc:/application/management/wbem:default SMC and WBEM Server
svc:/application/print/ipp-listener:default Internet Print Protocol Listening Service
svc:/application/print/ppd-cache-update:default ppd cache update
svc:/application/print/rfc1179:default BSD print protocol adapter
svc:/application/stosreg:default Service Tag OS Registry Inserter
svc:/application/x11/xfs:default X Window System font server
svc:/application/x11/xvnc-inetd:default X server that displays to VNC viewers
svc:/milestone/devices:default device configuration milestone
svc:/milestone/multi-user:default multi-user milestone
svc:/milestone/multi-user-server:default multi-user plus exports milestone
svc:/milestone/name-services:default name services milestone
svc:/milestone/network:default Network milestone
svc:/milestone/single-user:default single-user milestone
svc:/milestone/sysconfig:default Basic system configuration milestone
svc:/network/cde-spc:default CDE subprocess control
svc:/network/dns/client:default DNS resolver
svc:/network/finger:default finger
svc:/network/ftp:default FTP server
svc:/network/inetd:default inetd
svc:/network/initial:default initial network services
svc:/network/ipsec/ipsecalgs:default IPsec algorithm initialization
svc:/network/ipsec/policy:default IPsec policy initialization
svc:/network/iscsi/initiator:default -
svc:/network/login:rlogin remote login
svc:/network/loopback:default loopback network interface
svc:/network/nfs/cbd:default NFS callback service
svc:/network/nfs/client:default NFS client
svc:/network/nfs/mapid:default NFS ID mapper
svc:/network/nfs/nlockmgr:default NFS lock manager
svc:/network/nfs/rquota:default remote quota server
svc:/network/nfs/server:default NFS server
svc:/network/nfs/status:default NFS status monitor
svc:/network/pfil:default packet filter
svc:/network/physical:default physical network interfaces
svc:/network/routing-setup:default Initial routing-related configuration.
svc:/network/rpc-100235_1/rpc_ticotsord:default 100235
svc:/network/rpc/bind:default RPC bindings
svc:/network/rpc/cde-calendar-manager:default CDE calendar manager server
svc:/network/rpc/cde-ttdbserver:tcp ToolTalk database server
svc:/network/rpc/gss:default Generic Security Service
svc:/network/rpc/mdcomm:default SVM multi-node communications
svc:/network/rpc/meta:default SVM remote metaset services
svc:/network/rpc/metamed:default SVM remote mediator services
svc:/network/rpc/metamh:default SVM remote multihost disk services
svc:/network/rpc/rstat:default kernel statistics server
svc:/network/rpc/rusers:default network user name service
svc:/network/rpc/smserver:default removable media management
svc:/network/security/ktkt_warn:default Kerberos V5 warning messages daemon
svc:/network/service:default layered network services
svc:/network/shares/group:default Share Group
svc:/network/shares/group:zfs Share Group
svc:/network/shell:default rsh
svc:/network/smtp:sendmail sendmail SMTP mail transfer agent
svc:/network/ssh:default SSH server
svc:/network/stdiscover:default Service Tag discovery probe
svc:/network/stlisten:default Service Tag Discovery Listener
svc:/network/talk:default talk
svc:/network/telnet:default Telnet server
svc:/network/tnctl:default trusted networking templates
svc:/system/basicreg:default -
svc:/system/boot-archive:default check boot archive content
svc:/system/boot-archive-update:default update boot archive if necessary
svc:/system/console-login:default Console login
svc:/system/coreadm:default system-wide core file configuration
svc:/system/cron:default clock daemon (cron)
svc:/system/cryptosvc:default cryptographic services
svc:/system/device/fc-fabric:default Solaris FC fabric device configuration.
svc:/system/device/local:default Standard Solaris device configuration.
svc:/system/dumpadm:default system crash dump configuration
svc:/system/filesystem/autofs:default automounter
svc:/system/filesystem/local:default local file system mounts
svc:/system/filesystem/minimal:default minimal file system mounts
svc:/system/filesystem/root:default root file system mount
svc:/system/filesystem/usr:default read/write root file systems mounts
svc:/system/fmd:default Solaris Fault Manager
svc:/system/fpsd:default FP Scrubber - Online Floating Point Unit Test
svc:/system/identity:domain system identity (domainname)
svc:/system/identity:node system identity (nodename)
svc:/system/installupdates:default system update installer
svc:/system/keymap:default keyboard defaults
svc:/system/manifest-import:default service manifest import
svc:/system/name-service-cache:default name service cache
svc:/system/patchchk:default Launcher for Automatic Patching services
svc:/system/picl:default platform information and control
svc:/system/pkgserv:default Flush package command database to disk (see pkgadm(1m)).
svc:/system/postrun:default Postponed package postinstall command execution
svc:/system/power:default power management
svc:/system/resource-mgmt:default Global zone resource management settings
svc:/system/rmtmpfiles:default remove temporary files
svc:/system/sac:default SAF service access controller
svc:/system/scheduler:default default scheduling class configuration
svc:/system/svc/restarter:default master restarter
svc:/system/sysevent:default system event notification
svc:/system/sysidtool:net sysidtool
svc:/system/sysidtool:system sysidtool
svc:/system/system-log:default system log
svc:/system/utmp:default utmpx monitoring
svc:/system/webconsole:console java web console
svc:/system/zones:default Zones autoboot and graceful shutdown
Finding Failed Services
The Service Management Facility in Solaris 10 offers the ability to understand the relationship to other services.

v890/root# svcs -xv
svc:/application/print/server:default (LP print server)
State: disabled since Tue Feb 01 05:06:28 2011
Reason: Disabled by an administrator.
See: http://sun.com/msg/SMF-8000-05
See: man -M /usr/share/man -s 1M lpsched
Impact: 2 dependent services are not running:
svc:/application/print/rfc1179:default
svc:/application/print/ipp-listener:default
Fault Management Running via SMF
The Fault Management system is enabled through the Service Management Facility.

v890/root# svcs svc:/system/fmd:default
STATE STIME FMRI
online Feb_01 svc:/system/fmd:default
Fault Management System
Listing Fault Engines
The Fault Management infrastructure built into Solaris 10 monitors many of the core system features across architectures (both SPARC and Intel.) A listing of the current engines can be displayed via the "fmadm" command.

v890/root# fmadm config | sort
MODULE VERSION STATUS DESCRIPTION
cpumem-diagnosis 1.7 active CPU/Memory Diagnosis
cpumem-retire 1.1 active CPU/Memory Retire Agent
disk-transport 1.0 active Disk Transport Agent
eft 1.16 active eft diagnosis engine
ext-event-transport 0.1 active External FM event transport
fabric-xlate 1.0 active Fabric Ereport Translater
fmd-self-diagnosis 1.0 active Fault Manager Self-Diagnosis
fps-transport 1.0 active Solaris FP-Scrubber
io-retire 1.0 active I/O Retire Agent
snmp-trapgen 1.0 active SNMP Trap Generation Agent
sysevent-transport 1.0 active SysEvent Transport Agent
syslog-msgs 1.0 active Syslog Messaging Agent
zfs-diagnosis 1.0 active ZFS Diagnosis Engine
zfs-retire 1.0 active ZFS Retire Agent
Listing Faults on Platforms
The faults on a system can be listed.

v890/root# fmadm faulty
v890/root#
Solaris Diagostics
prtdiag

Not all diagnostics are managed through the Fault Management system via Service Management facility. There are still some hardware features which can only be seen via the Print Diagnostics command (such as fan speed.) The "prtdiag" command gives visibility to these components.

v890/user$ prtdiag -v
System Configuration: Sun Microsystems sun4u Sun Fire V890
System clock frequency: 150 MHz
Memory size: 32768 Megabytes

========================= CPUs ===============================================

Run E$ CPU CPU
Brd CPU MHz MB Impl. Mask
--- ----- ---- ---- ------- ----
A 0, 16 1500 32.0 US-IV+ 2.2
B 1, 17 1500 32.0 US-IV+ 2.1
A 2, 18 1500 32.0 US-IV+ 2.2
B 3, 19 1500 32.0 US-IV+ 2.1

========================= Memory Configuration ===============================

Logical Logical Logical
MC Bank Bank Bank DIMM Interleave Interleaved
Brd ID num size Status Size Factor with
---- --- ---- ------ ----------- ------ ---------- -----------
A 0 0 2048MB no_status 1024MB 8-way 0
A 0 1 2048MB no_status 1024MB 8-way 0
A 0 2 2048MB no_status 1024MB 8-way 0
A 0 3 2048MB no_status 1024MB 8-way 0
B 1 0 2048MB no_status 1024MB 8-way 1
B 1 1 2048MB no_status 1024MB 8-way 1
B 1 2 2048MB no_status 1024MB 8-way 1
B 1 3 2048MB no_status 1024MB 8-way 1
A 2 0 2048MB no_status 1024MB 8-way 0
A 2 1 2048MB no_status 1024MB 8-way 0
A 2 2 2048MB no_status 1024MB 8-way 0
A 2 3 2048MB no_status 1024MB 8-way 0
B 3 0 2048MB no_status 1024MB 8-way 1
B 3 1 2048MB no_status 1024MB 8-way 1
B 3 2 2048MB no_status 1024MB 8-way 1
B 3 3 2048MB no_status 1024MB 8-way 1

========================= IO Cards =========================

Bus Max
IO Port Bus Freq Bus Dev,
Brd Type ID Side Slot MHz Freq Func State Name Model
---- ---- ---- ---- ---- ---- ---- ---- ----- -------------------------------- ----------------------
I/O PCI 8 B 3 33 33 2,0 ok lpfc-pci10df,f900/sd (block) LP9002L
I/O PCI 9 B 5 33 33 3,0 ok fibre-channel-pci10df,f900.10df.+ LP9002L
I/O PCI 9 B 4 33 33 4,0 ok pci-pci8086,b154.0/network (netw+ PCI-BRIDGE
I/O PCI 9 B 4 33 33 0,0 ok network-pci108e,abba.11 SUNW,pci-ce/pci-bridge

No failures found in System
===========================

========================= Environmental Status =========================

System Temperatures (Celsius):
-------------------------------
Device Temperature Status
---------------------------------------
CPU0 60 OK
CPU1 54 OK
CPU2 55 OK
CPU3 53 OK
MB 24 OK
IOB 21 OK
DBP0 19 OK

=================================

Front Status Panel:
-------------------
Keyswitch position: NORMAL

System LED Status:
GEN FAULT REMOVE
[OFF] [OFF]

DISK FAULT POWER FAULT
[OFF] [OFF]

LEFT THERMAL FAULT RIGHT THERMAL FAULT
[OFF] [OFF]

LEFT DOOR RIGHT DOOR
[OFF] [OFF]

=================================

Disk Status:
Presence Fault LED Remove LED
DISK 0: [PRESENT] [OFF] [OFF]
DISK 1: [PRESENT] [OFF] [OFF]
DISK 2: [PRESENT] [OFF] [OFF]
DISK 3: [PRESENT] [OFF] [OFF]
DISK 4: [PRESENT] [OFF] [OFF]
DISK 5: [PRESENT] [OFF] [OFF]
DISK 6: [ EMPTY]
DISK 7: [ EMPTY]
DISK 8: [ EMPTY]
DISK 9: [ EMPTY]
DISK 10: [ EMPTY]
DISK 11: [ EMPTY]

=================================

Fan Bank :
----------

Bank Speed Status Fan State
( RPMS )
---- -------- --------- ---------
CPU0_PRIM_FAN 2000 [ENABLED] OK
CPU1_PRIM_FAN 2127 [ENABLED] OK
CPU0_SEC_FAN 0 [DISABLED] OK
CPU1_SEC_FAN 0 [DISABLED] OK
IO0_PRIM_FAN 3030 [ENABLED] OK
IO1_PRIM_FAN 2912 [ENABLED] OK
IO0_SEC_FAN 0 [DISABLED] OK
IO1_SEC_FAN 0 [DISABLED] OK
IO_BRIDGE_PRIM_FAN 3703 [ENABLED] OK
IO_BRIDGE_SEC_FAN 0 [DISABLED] OK

=================================

Power Supplies:
---------------
Current Drain:
Supply Status Fan Fail Temp Fail CS Fail 3.3V 5V 12V 48V
------ ------------ -------- --------- ------- ---- -- --- ---
PS0 GOOD 6 3 2 4
PS1 GOOD 6 3 2 4
PS2 GOOD 6 3 2 4

========================= HW Revisions =======================================

System PROM revisions:
----------------------
OBP 4.30.4 2009/08/19 07:21

IO ASIC revisions:
------------------
Port
Model ID Status Version
-------- ---- ------ -------
Schizo 8 ok 7
Schizo 9 ok 7

Sun/Oracle - Leading in Tape Storage


Sun/Oracle - Leading in Tape Storage

Oracle just released the best tape drive unit on the market today, for Government, Managed Services, and Enterprises.

Tape Systems

The old portable cassette players, with tapes, were very reliable when driving, jogging, or even playing at home. Older had disk based iPods would experience skips when one goes out jogging, but quickly started to put portable cassettes out of business. Of course, flash media is now replacing spinning fixed disk systems, but the capacity is not quite there to replace rotating fixed media for larger capacity systems.

Tape was the media of choice over the years for many reasons:
  • extremely high capacity
  • extreme long term media durability
  • extreme shock resistance
  • wide environmental operating factors
  • excellent portability
  • low cost
There is a reason why tape has been so widely used in the Space Program - the reliability of long term use on satellites and craft like the Space Shuttle.

Transferring data from a hard disk during a jog will cause a skip. Transferring data from a spinning disk under massive G-Force of a space craft launch, you are likely to get a crash.

Disks were getting more portable with the ability to auto-park heads to better absorb shock, disks could spin down to avoid shock issues, and the storage was surpassing tapes. Tape storage solution seemed to show little benefit in modern era.

StorageTek T10000C

With native storage on had disks topping out at 2 Terbytes, there seemed like little hope for tape.

The StorageTek T10000C was released from Oracle, who purchased Sun, who purchased StorageTek - the premier vendor of Tape Archive systems in data centers. This latest product turned back the clock on data center history:
  • 5 Terbyte performance
    (over doubling the maximum capacity of spinning rust on a fixed disk)
  • Built in Encryption
    (for securing of data on the cartridges)
  • Sustained 240 Megabytes / Second transfer rate
    (2x faster than copeting tape systems, 360MB/sec compressed transfer rate, out-performs inexpensive fixed-disk solutions)
  • Exabyte Storage Capacity in a Library
    (worlds largest tape library storage capacity)
  • WORM Capability
    (to provide auditing of systems in government compliance)
  • Extremely Energy Efficient
    (200x more energy efficient than low end disk arrays since tapes do not have to draw power to store data or remain spinning.)
  • Inexpensive Large Capacity Backups
    (up to15x less expensive than low-end disk arrays)
  • Long Life Expectancy
    (30+ years media archive life)
Network Management Connection

In an era where Network Management Centers are centralized and managing customers world-wide, governments require the interactions of system analysts to be archived and stored for long periods of time. Often, these interactions require video streaming from a desktop screen in a Windowing environment.

Writing this archive data to disk does not pass an audit, since someone can come along and delete a file. Encrypting the data becomes important, for long term storage. Massive media requirements are driven by screen video capture.

The StorageTek T10000C will meet the requirements of the strictest audit, the streaming throughput of the largest managed services center, the capacity for the highest definition monitors, and the lowest cost requirements of those large centers.

Don't miss your opportunity to simplify life in your managed services data center.

Tuesday, January 18, 2011

Sun Developer Days for NY/NJ: 2010-Dec

Sun Developer Days for NY/NJ: 2010-Dec

Abstract
Isaac Rozenfeld from Oracle/Sun posted an agenda and materials from a 2-day tour of New York City and Bridgewater tour of Solaris Days.

Agenda
08:30 Registration & Breakfast
09:00 Welcome Back, AgendaIsaac Rozenfeld [Audio] Focus on Financial Services - Ambreesh Khanna [Audio]
09:10 Solaris Networking Virtualization – Nicolas Droux [Audio]
10:00 Solaris Zones Update – Dan Price [Audio]
10:45 Image Packaging System – Bart Smaalders [Audio]
11:30 Platform Updates: x86 and SPARC – Sherry Moore [Audio]
12:15 Lunch, Isaac Rozenfeld's bonus session on running Solaris on top of the VirtualBox hypervisor [Audio]
01:00 Solaris Integration into Oracle – Damien Farnham [Audio]
01:45 Leaping Forward with Solaris Infiniband – David Brean [Audio]
02:30 Installation Experience Modernization – David Miner [Audio]
03:15 Oracle Enterprise Manager Ops Center – Mike Barrett [Audio]
04:00 Service Management Facility Architecture and Deployment – Liane Praza [Audio]
04:45 Q&A/Raffle

Executive Overview
Some of the important take-aways from a Network Management perspective.

10:00AM Solaris Zones Update by Dan Price
  • Page 5 - Older Solaris 8 & Solaris 9 SPARC physical machine (p2v) can be vitualized, as well as Linux under Intel
  • Page 8 - Security and Patch OS Updates can be made by merely migrating a zone containing an application from the old server to another server which had the patch applied
  • Page 24 - p2v support virtualizing Solaris 8, Solaris 9 (now Solaris 10 from a Solaris 11 platform); v2v for moving a zone between physical machines
  • Page 26 - Some common application support matrix where inquiries are constantly made
  • Page 19 - New "zonestat" command for quickly seeing health of components across multiple zones simultaneously.
10:45AM - Image Packaging System by Bart Smaalders
  • Pages 1-44 - Overview of the Solaris 11 Image Packaging System
11:30 AM - Platform Updates: x86 and SPARC by Sherry Moore
  • Page 4 - New SPARC T3 Processor (16 cores) image and features
  • Page 5 - I am tickled that Oracle used a SPARC diagram drawn by me (unfortunately they stretched it)
  • Page 6 - Current generation systems: images and features
  • 1:45PM - Leaping forward with Solaris Infiniband
  • Page 16 - Infiniband usage in Solaris Virtualized Zones Diagram
  • Page 30 - Important OS commands for Infiniband Fabric
2:30PM - Installation Experience Modernizations by David Miner
  • Page 4 - Solaris 10 and Solaris 11 Comparisons (important: Jumpstart Replaced!)
  • Page 5 - New Boot Environments based upon ZFS with "unlimited snapshots", breaking mirror with only one rollback is a thing of the past with Solaris 11
  • Page 9 - New Automated Installer Diagram, to replace Jumpstart… following pages illustrate use cases!
4:00PM - Service Management Facility Architecture and Deployment
  • Page 4 - Best Practices for deploying applicatons across networks
  • Page 7 - Best Practices for deploying applications onto ZFS
  • Page 9 - Software Support and Admin teams no longer require root or sudo with Solaris SMF for stop/start/restart
  • Page 11 - Application layer firewalls bundled as a service
  • Page 16 - Solaris 11 Image Packaging Sytem no longer uses scripts, but bundles into SMF
  • Page 17 - Automatic Fault notifications through SMF via email & SNMP
  • Page 19 - Best Practices of modern virtualized Solaris Application Deployment

Tuesday, December 21, 2010

Sun Founders Panel 2006

Sun Founders Panel 2006

This video from the Computer History Museum contains an intriguing panel with Sun founders and pioneers Andy Bechtolsheim, Bill Joy, Vinod Khosla, Scott McNealy, and John Gage.



A memorable quote from the session, "Get them on tape before they die." Some of the details surrounding this session are located here. This video is a "must watch" for anyone involved in the technology business.

Technologist David Halko states: predicting the future today requires understanding the past.

Sunday, December 5, 2010

CoolThreads UltraSPARC and SPARC Processors


[UltraSPARC T3 Micrograph]

CoolThreads UltraSPARC and SPARC Processors

Abstract:

Processor development takes an immense quantity of time, to architect a high-performance solution, and an uncanny vision of the future, to project market demand and acceptance. In 2005, Sun embarked on a bold path moving toward many cores and many threads per core. Since the purchase of Sun by Oracle, the internal SPARC road map from Sun had clarified.


[UltraSPARC T1 Micrograph]
Generation 1: UltraSPARC T1
A new family of SPARC processors was announced by Sun on 2005 November 14.
  • Single die
  • Single socket
  • 64 bits
  • 4, 6, 8 integer cores
  • 4, 6, 8 crypto cores
  • 4 threads/core
  • 1 shared floating point core
  • 1.0 GHz - 1.4 GHz clock speed
  • 279 million transisters
  • 378 mm2
  • 90 nm CMOS (TI)
  • 1 JBUS port
  • 3 Megabyte Level 2 Cache
  • 1 Integer ALU per Core
  • ??? Memory Controllers
  • 6 Stage Integer Pipeline per Core
  • No embedded Ethernet into CPU
  • Crypto Algorithms: ???
Platform designed as a front-end server for web server applications. With a massive number of cores, it was designed to provide web-tier performance similar to existing quad-socket systems leveraging a single socket.

To understand the ground-breaking advancement in this technology, most processors were single core, with an occasional dual core processor (with cores glued together through a more expensive process referred to as a multi-chip module, driving higher software licensing costs for those platforms.)


Generation 2: UltraSPARC T2
The next generation of the CoolThreads processor was announced by Sun on 2007 August.
  • Single die
  • Single Socket
  • 64 bits
  • 4, 6, 8 integer cores
  • 4, 6, 8 crypto cores
  • 4, 6, 8 floating point units
  • 8 threads/core
  • 1.2 GHz - 1.6 GHz clock speed
  • 503 million transisters
  • 342 mm2
  • 65 nm CMOS (TI)
  • 1 PCI Express port (1.0 x8)
  • 4 Mageabyte Level 2 Cache
  • 2 Integer ALU per Core
  • 4x Dual Channel FBDIMM DDR2 Controllers
  • 8 Stage Integer Pipeline per Core
  • 2x 10 GigabitEthernet on-CPU ports
  • Crypto Algorithms: DES, Triple DES, AES, RC4, SHA1, SHA256, MD5, RSA-2048, ECC, CRC32
This processor was designed for higher compute intensive requirements and incredibly efficient network capacity. Platform made an excellent front-end server for applications as well as Middleware, with the ability to do 10 Gigabit wire-speed encryption with virtually no CPU overhead.

Competitors started to build Single-Die dual-core CPU's with Quad-Core processors by gluing dual-core processors into a Multi-Chip Module.


[UltraSPARC T2 Micrograph]
Generation 3: UltraSPARC T2+
Sun quickly released the first CoolThreads SMP capable UltraSPARC T2+ in 2008 April.
  • Single die
  • 1-4 Sockets
  • 64 bits
  • 4, 6, 8 integer cores
  • 4, 6, 8 crypto cores
  • 4, 6, 8 floating point units
  • 8 threads/core
  • 1.2 GHz - 1.6 GHz clock speed
  • 503 million transisters
  • 342 mm2
  • 65 nm CMOS (TI)
  • 1 PCI Express port (1.0 x8)
  • 4 Megabyte Level 2 Cache
  • 2 Integer ALU per Core
  • 2x? Dual Channel FBDIMM DDR2 Controllers
  • 8? Stage Integer Pipeline per Core
  • No embedded Ethernet into CPU
  • Crypto Algorithms: DES, Triple DES, AES, RC4, SHA1, SHA256, MD5, RSA-2048, ECC, CRC32
This processor allowed the T processor series to move from the Tier 0 web engines and Middleware to Application tier. Architects started to understand the benefits of this platform entering the Database tier. This was the first Coolthreads processor to scale past 1 and up to 4 sockets.

By this time, competition really started to understand that Sun had properly predicted the future of computing. The drive toward single-die Quad-Core chips have started with Hex-Core Multi-Chip Modules being predicted.


Generation 4: SPARC T3
The market became nervous with Oracle purchasing Sun. The first Oracle branded CoolThreads SMP capable UltraSPARC T3 was launched in in 2010 September.
  • Single die
  • 1-4 Sockets
  • 64 bits
  • 16 integer cores
  • 16 crypto cores
  • 16 floating point units
  • 8 threads/core
  • 1.67 GHz clock speed
  • ??? million transisters
  • 377 mm2
  • 40 nm
  • 2x PCI Express port (2.0 x8)
  • 6 Megabyte Level 2 Cache
  • 2 Integer ALU per Core
  • 4x DDR3 SDRAM Controllers
  • 8? Stage Integer Pipeline per Core
  • 2x 10 GigabitEthernet on-CPU ports
  • Crypto Algorithms: DES, 3DES, AES, RC4, SHA1, SHA256/384/512, Kasumi, Galois Field, MD5, RSA to 2048 key, ECC, CRC32
This processor was more than what the market was anticipating from Oracle. This processor took all the features of the T2 and T2+ combined them into the new T3 with an increase in overall features. No longer did the market need to choose between multiple sockets or embedded 10 GigE interfaces - this chip has it all plus double the cores.

The market, immediately before this release, the competition was releasing single die hex-core and octal-core CPU's using multi-chip modules, by gluing them together. The T3 was a substantial upgrade over the competition by offering double the cores on a single die.


Generation 5: SPARC T4
Oracle indicated in December 2010 that they had thousands of these processors in the lab and predicted this processor will be released end of 2011.

After the announcement, a separate press release indicated processors will have a renovated core, for higher single threaded performance, but the socket will offer half the cores.

Most vendors are projected to have 8 core processors available (through Multi-Chip modules) by the time the T3 is released, but only the T4 should be on a single piece of silicon during this period.


[2010-12 SPARC Solaris Roadmap]
Generation 6: SPARC T5

Some details on the T5 were announced with the T4. Processors will use the renovated T4 core, with a 28nm process. This will return to 16 cores per socket again. This processor may be the first Coolthreads T processor able to scale from 1-8 processors. It is projected to appear in early 2013.

Some vendors are projecting to have 12 core processors on the market using Multi-Chip Module technology, but when the T5 is released, this should still be the market leader in 16 cores per socket.

Network Management Connection

Consolidating most network management stations in a globalized environment works very well with the Coolthreads T-Series processors. Consolidating multiple slower SPARC platforms onto single and double socket T series have worked well over the past half decade.

While most network management polling engines will scale linearly with these highly-threaded processors, there are some operations which are bound to single threads. These type of processes include event correlation, startup time, and syncronization after a discovery in a large managed topology.

The market will welcome the enhanced T4 processor core and the T5 processor, when it is released.

Friday, December 3, 2010

Scalable Highest Performing Clusters at Value Pricing



Scalable Highest Performing Clusters at Value Pricing

Abstract:
Oracle presented another milestone achievement in their 5 year SPARC/Solaris road map with Fujitsu. John Fowler stated: "Hardware without Software is a Door-Stop, Solaris is the gateway."

High-Level:
The following is a listing of my notes from the two sessions. The notes have been combined, with Larry Ellison outlining the high-level and John Fowler presenting the lower-level details. SPARC T3 making world-record benchmarks. New T3 based integrated products. Oracle's Sun/Fujitsu M-Series gets a speed bump. SPARC T4 is on the way.

Presentation Notes:


New TpmC Database OLTP Performance
  • SPARC Top cluster performance
  • SPARC Top cluster price-performance
  • (turtle)
    HP Superdome Itanium 4 Million Transactions/Minute
  • (stallion)
    IBM POWER7 Power 780 10 Million Transactions/Minute
    (DB2 clustered through custom applications)
  • Uncomfortable 4 month for Oracle, when IBM broke the Oracle record
  • (cheetah)
    Sun SPARC 30 Million Transactions/Minute
    (standard off-the-shelf Oracle running RAC)
  • Oracle/Sun performance benchmark => ( IBM + HP ) x 2 !
  • Sun to IBM Comparison:
    3x OLTP Throughput, 27% better Price/Performance, 3.2x faster response time
  • Sun to HP Comparison:
    7.4x OLTP Throughput 66 Better Price/Performance, 24x compute density
  • Sun Supercluster:
    108 sockets, 13.5 TB Memory, Infiniband 40 Gigabit link, 246 Terabytes Flash, 1.7 Petabytes Storage, 1 Quadrillion rows, 43 Trillion transactions per day, 0.5 sec avg response

New Gold Release
  • Gold Standard Configurations are kept in the lab
  • What the customer has, the support organization will have assembled in the lab
  • Oracle, Sun, Cisco, IBM will all keep their releases and bug fixes in sync with releases

SPARC Exalogic Elastic Cloud
  • Designed to run Middleware
  • New T3 processor based
  • 100% Oracle Middleware is Pure Java
  • Tuned for Java and Oracle Fusion Middleware
  • Load-balances with elasticity
  • Ships Q1 2011
  • T3-1B SPARC Compute Blades based
    30 Compute Servers, 16 cores/server, 3.8 TB RAM, 960 GB mirrored flash disks, 40 TB SAS Storage, 4 TB Read Cache, 72 GB Write Cache, 40 Gg/sec Infiniband, 10 GigE to Datacenter

SPARC Supercluster
  • New T3 processor based and M processor based
  • T3-2 = 2 nodes, 4 CPU's, 64 cores/512 threads, 0.5 TB RAM, 96 TB HDD ZFS, 1.7TB Write Flash, 4TB Read Flash, 40 Gbit Infiniband
  • T3-4 = 3 nodes, 12 CPU's, 192 cores/1536 threads, 1.5 TB RAM, 144 TB HDD ZFS, 1.7TB Write Flash, 4TB Read Flash, 40 Gbit Infiniband
  • M5000 = 2 nodes, 16 CPU's, 64 core/128 threads, 1 TB RAM, 144 TB HDD ZFS, 1.7TB Write Flash, 4TB Read Flash, 40 Gbit Infiniband

T3 Processor in production
  • Releases already, performing in these platforms
  • 1-4 processors in a platform
  • 16 cores/socket, 8 threads/core
  • 16 crypto-engines/socket
  • More cores, threads, 10 GigE on-chip, more crypto engines

T4 Processor in the lab!
  • Thousands under test in the lab, today
  • To be released next year
  • 1-4 processors
  • 8 cores/socket, 8 threads/core
  • faster per-thread execution

M3 Processor from Fujitsu
  • SPARC VII+
  • 1-64 SPARC64 VII+ Processors
  • 4 cores, 2 threads/core
  • Increased CPU frequency
  • Double cache memory
  • 2.4x performance of original SPARC64 VI processor
  • VII+ boards will slot into the VI and VII board chassis
Flash Optimization
- Memory hierarchy with software awareness

Infiniband
- Appropriate for High Performance Computing
- Dramatically better performance than Ethernet for linking servers to servers & storage

New Solaris 11 Release

  • Next Generation Networking
    re-engineered network stack
    low latency high bandwidth protocols
    virtualized
  • Cores and Threads Scale
    Adaptive Thread and Memory Placement
    10,000's of core & threads
    thread observability with DTrace

  • Memory Scale
    Dynamic optimization for large memory configs
    Advanced memory placement
    VM systems for 1000's TB memory configs

  • I/O Performance
    Enhanced NUMA I/O framework
    Auto-Discovery of NUMA architecture
    I/O resources co-located with CPU for scale/performance

  • Data Scale
    ZFS Massive storage for massive datasets

  • Availability
    Boot times in seconds
    Minimized OS Install
    Risk-Free Updates with lightweight boot and robust package dependency
    Extensive Fault Management with Offline failing components
    Application Service Managemment with Restart failed applications and associated services quickly

  • Security
    Secure by default
    Secure boot validated with onboard Trusted Platform Module
    Role Based Root Access
    Encrypted ZFS datasets
    Accelerated Encryption with hardware encryption support

  • Trusted Solaris Extensions
    Dataset labels for explicit access rules
    IP labels for secure communication

  • Virtualization
    Network Virtualization to add to Server and Storage Virtualization
    Network Virtualization includes Virtual NIC's and Virtual Switches
SPARC Supercluster Architecture
  • Infiniband is 5x-8x faster than most common Enterprise interconnects
    Infiniband has been leveraged with storage and clustering in software
  • Flash is faster than Rotating Media
    Integrated into the Memory AND Storage Hierarchy

SPARC 5 Year Roadmap
  • SPARC T3 delvered in 2010
  • SPARC VII+ delivered in 2010
  • Solaris 11 and SPARC T4 to be delivered in 2011
Next generation of mission critical enterprise computing
  • Engineer software with hardware products
  • Deliver clusters for general purpose computing
  • Enormous levels of scale
  • Built in virtualization
  • Built in Security
  • Built in management tools
  • Very Very high availability
  • Tested with Oracle software
  • Supported with Gold Level standard
  • Customers spend less time integrating and start delivering services on systems engineered with highest performance components