Sunday, October 17, 2010

FPing: Options & Tuning

FPing: Options & Tuning


The FPing command offers substantial capability in polling multiple devices by polling asyncronously. FPing is projected to be bundled with Solaris 11, a worthy tool to be added to the Solaris toolkit. There are a lot of command line options, which various manual pages & elp files hold incomplete or conflicting information. This document is an attempt to clarify the options.

FPing Version:

The following illustrates the version of "fping" which this commentary is used for:

sunt2000$ fping -v
fping: Version 2.4b2_to $Date: 2001/01/25 11:25:04 $
fping: comments to

This version is currently installed via an SVR4 package from sunfreeware and can be downloaded under Solaris 10 here.

Issues Experienced:

A combination of selected command line arguments, total number of devices, and delay in the response from the devices can occasionally cause a crash of "fping" with the error "Arithmetic Exception".

The individual maintaining the fping source code has not been responsive to requests for clarifications regarding the package he has been maintaining regarding various crashes which have been experienced with the package. After working on the crash issue for several weeks, it became necessary to clarify the command line options and publish a short blog on the experience.

Command Line Options:

The command line options below were taken from the manual page for the Solaris packaged distribution and augmented with additional comments. Small fonts in parenthesis are original manual page entries, italics represent augmented description.

fping [ options ] [ systems... ]

-a Show systems that are alive.

-A Display targets by address rather than (DNS name) operating system name resolution.

-b n Number of bytes of ping data to send. The minimum size (normally 12) allows room for the data that fping needs to do its work (sequence number, timestamp). The reported received data size includes the IP header (normally 20 bytes) and ICMP header (8 bytes), so the minimum total size is 40 bytes. Default is 56, as in ping. Maximum is the theoretical maximum IP datagram size (64K), though most systems limit this to a smaller, system-dependent number.

-B n In the default mode, fping sends several requests to a target before giving up, waiting longer for a reply on each successive request. This parameter is the value by which the wait time is multiplied on each successive request; it must be entered as a floating-point number (x.y). This is referred to as an Exponential Backoff Factor. The default is 1.5.

-c Number of request packets to send to each target. In this mode, a line is displayed for each received response (this can suppressed with -q or -Q). Also, statistics about responses for each target are displayed when all requests have been sent (or when interrupted). The default of 1.

-C Similar to -c, but the per-target statistics are displayed in a format designed for automated response-time statistics gathering. The output display is also called Verbose Mode. For example:
% fping -C 5 -q somehost
somehost : 91.7 37.0 29.2 - 36.8
shows the response time in milliseconds for each of the five requests, with the "-" indicating that no response was received to the fourth request.

-d Use (DNS to lookup) operating system name resolution lookup on address of return ping packet. This allows you to give fping a list of IP addresses as input and print hostnames in the output.

-e Show elapsed (round-trip) time of packets.

-f file Read list of targets from a file. This option can only be used by the root user. Not used when -g is not specified. Regular users should pipe in the file via stdin:
% fping <>-g Generate a target list from a supplied IP netmask, or a starting and ending IP. Specify the netmask or start/end in the targets portion of the command line.
ex. To ping the class C 192.168.1.x, the specified command line could look like either:
fping -g
fping -g

-h Print usage message.

-i n The minimum amount of time (in milliseconds) between sending a ping packet to any target (default is 25). This is the icmp packet sending interval. The poller will move linearly through the list of provided hosts or ip addresses, waiting this interval after sending a packet before sending a packet to the next host or ip in the list. For large quantity of nodes, this number may need to be reduced to avoid a crash with an "Arithmetic Exception" error. Some networks may drop packets if this is set too low. Maintainer web site manual page specifies the default value is 10. If this value is critical for your implementation, specify it.

-l Loop sending packets to each target indefinitely. Can be interrupted with ctl-C; statistics about responses for each target are then displayed. May not die in looping mode if process reading STDOUT is closed abnormally.

-m Send pings to each of a target host's multiple interfaces.

-n Show target by operating system resolved name. Same as -d.

-p n In looping or counting modes (-l, -c, or -C), this parameter sets the time in milliseconds that fping waits between successive packets to an individual target. Useful in unreliable networks for spreading retry a distance from former attempt. For large quantity of nodes, increasing this number may help reduce instances of a crash with an "Arithmetic Exception" error. Default is 1000.

-q Quiet. Don't show per-target results, just set final exit status.

-Q n Like -q, but show summary results every n seconds. If this summary happens before all the devices can be polled, an "Arithmetic Exception" error may occur. This value may need to be increased, to aleviate this crashing symptom.

-r n Retry limit (default 3). This is the number of times an attempt at pinging a target will be made, not including the first try.

-s Print cumulative statistics upon exit.

-t n Initial target timeout in milliseconds (default 500). In the default mode, this is the amount of time that fping waits for a response to its first request. Successive timeouts are multiplied by the backoff factor.

-u Show targets that are unreachable.

-v Print fping version information.

Thursday, October 14, 2010

Solaris 10: History of Zones

There was a great article holding a short the history of Zones under Solaris 10 on a version by version release.
Solaris 10 11/06 (Update 3)
Zone renaming
Zone move and clone
zone attach/detach
Privileges in zone configuration

Solaris 10 8/07 (Update 4)
Upgrades, Live upgrades (ZULU)
IP Instances (dedicated NIC/separate TCP/IP stack)
Resource setting for memory/CPU

Solaris 10 5/08 (Update 5)
Dry run of zone migration (zoneadm -n)
CPU caps for zones

Solaris 10 10/08 (Update 6)
Update on attach
Default router in shared stack
Full ZFS support for zones

Solaris 10 10/09 (Update 8)
Turbo-Charged SVR4 Packaging
Zones Parallel Patching

Solaris 10 9/10 (Update 9)
Zones P2V (Physical to Virtual)
Host ID Emulation
"Upgrade on attach"
The benefits to Zones are many, but a few include: zero cost, incredible density on a single OS instance (up to 4000), and virtually no overhead.

Solaris Zones are an essential part of any cost effective data center when performance managed services for external customers.

Solaris 11 Express 2010.11 and ZFS

Solaris 11 Express 2010.11 and ZFS

There was much discussion over the years over the discontinuance of Solaris Express distribution, the creation of OpenSolaris distribution, the discontinuance of OpenSolaris distribution, and ultimately the re-announcement of Solaris Express distribution. More information regarding Solaris Express has been leaked.

Designated Release:
It appears that there will be a release of Solaris Express referred next month or so, designated as Solaris 11 Express 2010.11!

RAID-Z/Mirror Hybrid Allocator:
Mirrored layout across the children of a RAID-Z vdev to ensure latency sensitive metadata can be read in a single I/O. See bug 6977913 reference.

ZFS "zdb" Enhancements:
Support for decompression, checksumming, and RAID-Z in "zdb" has been requested since 2008-Q4. See bug 6757444 reference.

ZFS File System Listing Enhancement:
Performance has been improved for listing ZFS file systems.

ZFS Encryption:
During 2003, a request was made to include encryption in ZFS, a welcome addition for laptops. The bug-id for this feature is 4854202.

Future Solaris 11 Support:
Solaris 11 is expected to power the new generation of embedded systems from Oracle including Exadata X2-8 database machine and Exalogic cloud-in-a-box!

There is more than this in the next release to come, but this gives us a clear direction of where we are at and where we are going!

Tuesday, October 5, 2010

US Department of Energy: No POWER Upgrade From IBM

US Department of Energy: No POWER Upgrade From IBM


Some ay no one was ever fired for buying IBM, but no government or business ever got trashed for buying SPARC. The United States Department of Energy bought an IBM POWER system with no upgrade path and no long term spare parts.

[IBM Proprietary POWER Multi-Chip Module]


The U.S. Depertmant of Energy purchased a petaflops-class hybrid blade supercomputer called the IBM "Roadrunner" that performed into the multi-petaflop range for nuclear simulations at the Los Alamos National Laboratory. It was based upon the IBM Blade platform. Blades were based upon an AMD Opteron and hybrid IBM POWER / IBM Cell architecture. A short article was published in October 2009 in The Register.

Today's IBM:

A month later, the super computer was not mentioned at the SC09 Supercomputin Trade Show at Oregon, because IBM killed it. Apparently, it was killed off 18 months earlier - what a waste of American tax payer funding!

Tomorrow's IBM:

In March 2010, it was published that IBM gave it's customers (i.e. the U.S. Government) three months to buy spares, because future hybrid IBM POWER / Cell products were killed. Just a few months ago, IBM demonstrated their trustworthlessness with their existing Thin Client customers and partners by abandoning their thin client partnership and using the existing partner to help fund IBM's movement to a different future thin client partner!

Obama Dollars:

It looks like some remaining Democratic President Obama stimulus dollars will be used to buy a new super computer from Cray and cluster from SGI. The mistake of buying IBM was so huge that it took a massive spending effort from the Federal Government to recover from losing money on proprietary POWER.

[Fujitsu SPARC64 VII Processor]

[Oracle SPARC T3 Processor]
Lessons Learned:
If only the U.S. Government did not invest in IBM proprietary POWER, but had chosen an open CPU architecture like SPARC, which offers two hardware vendors: Oracle/Sun and Fujitsu.

[SUN UltraSPARC T2; Used in Themis Blade for IBM Blade Chassis]

Long Term Investment:

IBM POWER is not an open processor advocated by other systems vendor. Motorola abandoned the systems market for POWER from a processor production standpoint. Even Apple abandoned POWER on the desktop & server arena. One might suppose that when IBM kills a depended upon product, that one could always buy video game consoles and place them in you lights-out data center, but that is not what the Department of Energy opted for.

Oracle/Sun has a reputation of providing support for systems a decade old, and if necessary, Open SPARC systems and even blades for other chassis can be (and are) built by other vendors(i.e. Themis built an Open SPARC blade for an IBM Blade chassis.) SPARC processors have been designed & produced by different processor and system vendors for over a decade and a half. SPARC is a well proven long term investment in the market.

Network Management Connection:

If you need to build a Network Operation Center, build it upon the infrastructure the global telecommunications providers had trusted for over a decade: SPARC & Solaris. One will not find serious network management applications on IBM POWER, so don't bother wasting time looking. There are reasons for it.

Monday, October 4, 2010

DTrace: Managing Applications in Modern Operating Systems

DTrace: Managing Applications in Modern Operating Systems

Abstract: Modern operating systems often have many commands at different layers in the software stack which are required in order to debug issues. DTrace changes this and provides a secure single interface to investigate nearly every layer in the software stack in a running production system, where developers can even create their own hooks for when they may need additional future insight.

Video interview on DTrace:

PDF presentation on DTrace:

The DTrace book:

Disk monitoring using DTrace:

Least Privilege with DTrace - No root required: