Solaris 11: The pkg Repositories

[Solaris Logo, courtesy former Sun Microsystems]

Abstract:

Packaging has long been the basis of modern Operating Systems, dating back to AT&T System V. Solaris adopted SVR4 packaging, when Sun Microsystems started growing from an Operating System needing a compiler to a production Operating System to be deployed. SVR4 Packaging was originally based upon the concept of a Stream (recorded upon Sequential Block infrastructures likes Tape) or a Tree (recorded Random Block infrastructures like Disk.) Sun Microsystems was astutely aware that "http" protocol was not much different from a "tape", where a stream of data was pulled down, and they upgraded SVR4 to support HTTP repositories with encryption and license keys. Somewhere along the way, Sun lost their way, and created a proprietary packaging system with fewer capabilities, called IPS, based upon the new command "pkg"... but Oracle is making the best of it.

[former OpenSolaris logo]

The "pkg" Repository

The concept of a Package Repository with the Image Packaging System was introduced with OpenSolaris. The repository would be served up through a web server and secured with certificates.

[Oracle Logo, courtesy Oracle Corporation]

Oracle pkg Repositories

There are two kinds of Oracle “pkg” repositories:

1.       Non-production Release Repository
Designated as: http://pkg.oracle.com/solaris/release/

2.       Production Support Repository
Designated as: https://pkg.oracle.com/solaris/support/

The document describing getting access to the support repository:
http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-018-howto-update-s11-1572261.html

The document describing the Solaris 11.2 Package Publisher info:
http://docs.oracle.com/cd/E36784_01/html/E36802/gijmo.html

Checking Repository

The newly installed OS is using the Oracle Package Publisher defaults to the Release Repository.

sun9876/root# pkg publisher

PUBLISHER                   TYPE     STATUS P LOCATION                           

solaris                     origin   online F http://pkg.oracle.com/solaris/release/

Additional detail can be reviewed about Oracle’s “solaris” release publisher:

sun9876/root# pkg publisher solaris

            Publisher: solaris

                Alias:

           Origin URI: http://pkg.oracle.com/solaris/release/

              SSL Key: None

             SSL Cert: None

          Client UUID: 6367a630-fbe6-11e3-8701-5bf522237f54

      Catalog Updated: August 18, 2015 04:44:20 PM

              Enabled: Yes

To check the current OS Release and Update – note: installed is Solaris 11.2 (0.175 is Solaris 11) SRU 0

sun9876/root# pkg info entire

          Name: entire

       Summary: Incorporation to lock all system packages to the same build

   Description: This package constrains system package versions to the same

                build.  WARNING: Proper system update and correct package

                selection depend on the presence of this incorporation.

                Removing this package will result in an unsupported system.

      Category: Meta Packages/Incorporations

         State: Installed

     Publisher: solaris

       Version: 0.5.11

 Build Release: 5.11

        Branch: 0.175.2.0.0.42.0

Packaging Date: June 24, 2014 07:38:32 PM

          Size: 5.46 kB

          FMRI: pkg://solaris/entire@0.5.11,5.11-0.175.2.0.0.42.0:20140624T193832ZZ

To check the Oracle Release Repository – note: available is Solaris 11.2 (0.175 is Solaris 11) SRU 1

sun9876/root# pkg info -r entire

          Name: entire

       Summary: Incorporation to lock all system packages to the same build

   Description: This package constrains system package versions to the same

                build.  WARNING: Proper system update and correct package

                selection depend on the presence of this incorporation.

                Removing this package will result in an unsupported system.

      Category: Meta Packages/Incorporations

         State: Not installed

     Publisher: solaris

       Version: 0.5.11

 Build Release: 5.11

        Branch: 0.175.2.1.0.2.1

Packaging Date: September 23, 2014 10:49:40 PM

          Size: 5.46 kB

          FMRI: pkg://solaris/entire@0.5.11,5.11-0.175.2.1.0.2.1:20140923T224940Z

There are 148 updates available from the Oracle Release repository.

sun9876/root# pkg list -u | wc -l

     148

To list the updates available:

sun9876/root# pkg list -u | head

NAME (PUBLISHER)                                  VERSION                    IFO

archiver/gnu-tar                                  1.27.1-0.175.2.0.0.42.1    i--

compress/bzip2                                    1.0.6-0.175.2.0.0.42.1     i--

compress/gzip                                     1.5-0.175.2.0.0.42.1       i--

compress/p7zip                                    9.20.1-0.175.2.0.0.42.1    i--

compress/pbzip2                                   1.1.6-0.175.2.0.0.42.1     i--

compress/pixz                                     1.0-0.175.2.0.0.42.1       i--

compress/unzip                                    6.0-0.175.2.0.0.42.1       i--

compress/xz                                       5.0.1-0.175.2.0.0.42.1     i--

compress/zip                                      3.0-0.175.2.0.0.42.1       i—

A dry-run of the update shows 8 packages updates available with release, and size - no reboot required.

sun9876/root# pkg update -nv

            Packages to update:         8

     Estimated space available: 275.69 GB

Estimated space to be consumed:  65.63 MB

       Create boot environment:        No

Create backup boot environment:       Yes

          Rebuild boot archive:        No

Changed packages:

solaris

  consolidation/sunpro/sunpro-incorporation

    0.5.11,5.11-0.175.2.0.0.37.0:20140414T130238Z -> 0.5.11,5.11-0.175.2.1.0.4.0:20140728T200719Z

  consolidation/userland/userland-incorporation

    0.5.11,5.11-0.175.2.0.0.42.1:20140623T010405Z -> 0.5.11,5.11-0.175.2.1.0.2.0:20140723T184045Z

  developer/assembler

    0.5.11,5.11-0.175.2.0.0.37.0:20140414T130241Z -> 0.5.11,5.11-0.175.2.1.0.4.0:20140728T200720Z

  entire

    0.5.11,5.11-0.175.2.0.0.42.0:20140624T193832Z -> 0.5.11,5.11-0.175.2.1.0.2.1:20140923T224940Z

  system/library/c++-runtime

    0.5.11,5.11-0.175.2.0.0.37.0:20140414T130401Z -> 0.5.11,5.11-0.175.2.1.0.4.0:20140728T200722Z

  system/library/math

    0.5.11,5.11-0.175.2.0.0.37.0:20140414T130409Z -> 0.5.11,5.11-0.175.2.1.0.4.0:20140728T200728Z

  system/library/mmheap

    0.5.11,5.11-0.175.2.0.0.23.0:20130916T153150Z -> 0.5.11,5.11-0.175.2.1.0.4.0:20140728T200732Z

  system/library/openmp

    0.5.11,5.11-0.175.2.0.0.37.0:20140414T130412Z -> 0.5.11,5.11-0.175.2.1.0.4.0:20140728T200733Z

To update from 11.2.0 to 11.2.1 (which was the latest at the time of this article publishing):

sun9876/root# pkg update

Management through Ops Center

If the operating system instance is managed through Ops Center, the publisher repositories are changed, and patching can be done centrally.

Ops Center server is the local proxy, holding patches & packages from Oracle

sun5582/dh127087$ pkg publisher

PUBLISHER                   TYPE     STATUS P LOCATION                           

solaris                     origin   online F https://oracle-oem-oc-mgmt-sun9999:8002/IPS/

cacao                       origin   online F https://oracle-oem-oc-mgmt-sun9999:8002/IPS/

mp-re          (non-sticky) origin   online F https://oracle-oem-oc-mgmt-sun9999:8002/IPS/

opscenter                   origin   online F https://oracle-oem-oc-mgmt-sun9999:8002/IPS/

The operating systems managed through Ops Center can be patched remotely or can be patched through the command line, using Ops Center server as the supported package repository.

Conclusions

While the detour that Sun Microsystems took, taking packaging back a couple decades, Oracle started to make the best of it. With the release of Ops Center, to manage the Solaris cloud components to automatically configure the pkg components and provide a continuous feed of packages for their Operating System and Firmware, Oracle has been making some sweet lemonade from their lemons.

Joyent: Encapsulating Linux through Docker into a Zone

[Solaris 11 Launch image, courtesy Oracle]

Abstract:

Virtualization has been available in the UNIX OS world. The creation of users in a time sharing environment, to isolate executable threads from one another as well as protect files in an underlying file system started the journey. The creation of the Virtual File System, where disks could me mounted anywhere in a file system tree (instead of drive letter) revolutionized computing to allow those systems to grow in the shared environment! The creation of "chroot" so an application could run in it's own file system space, made an application "feel" like it is on a dedicated system. The merging of SVR4 into Solaris created a robust multi-processor infrastructure to host multi-user and  multi-tenant systems. The creation of Zones under SVR4 Solaris 10, further extrapolated the original concepts of the UNIX "chroot", isolating CPU, Memory, Users, Storage - effectively making a single instance of the Solaris OS truly multi-tenant. The creation of Branded Zones for Linux and Solaris came later, offering entire operating systems to be encapsulated under Intel and SPARC Solaris systems. Newer proprietary technologies continue to enter the horizon.

[Oracle Linux, courtesy Oracle]

The Linux Problem

People participating in the Linux ecosystem are interested in creating new raw environments,  isolated to their operating system under proprietary Intel processors, to supply a reasonable replacement for mature infrastructure. These replacements constitute very long efforts, which often never really get completed. Veterans understand the benefit of good engineering and can often take systems "to the next level." Vendors like Oracle had taken Linux, ran their applications on top of it, and supplied the patches necessary to keep Linux stable.

Joyent: Zones(KVM and Linux)

Former employees of Sun Microsystems continue to do the heavy lifting in the industry. Network Management wrote about Joyent's efforts to port KVM into Solaris Zones under their SmartOS, based upon Illumos. Illumos originated from Sun Microsystem's OpenSolaris project (which became the basis of Oracle's Solaris 11.)

[Solaris Zone/Container concept, courtesy former Sun Microsystems]

Joyent: Zones(Docker and Linux)

One might expect that Cloud companies who are obsessed with Virtualization like Joyent would continue their quest for a "better cloud". In 2015, Joyent released a presentation on the porting of Docker to encapsulate Linux into a Zone... using the same SmartOS based upon Illumos, which found it's roots in Sun Microsystem's OpenSolaris.

Docker and the Future of Containers in Production from bcantrill

For Joyent, The Cloud means chasing every container technology and integrating it into SmartOS, to give their customers choice, while simultaneously utilizing their infrastructure as efficiently as possible.

Conclusion

SVR4 UNIX and Sun Solaris developers have a long history of virtualization. The success story of Joyent in "Cloud" environments continues to lead the market in vision, taking things which were good but raw, and rolling them into mature facilities which continues to make the computing industry grow!

Available in PDF: Bell System Technical Journal, 1922-1983

Alcatel-Lucent released the entire Bell System Technical Journal catalog in PDF format. http://www.alcatel-lucent.com/bstj/

With this posting of the Bell System Technical Journal from volume 1 issue 1 in July 1922 to the final issue published in December 1983, we are pleased to be able to open the vault of this knowledge to our global technical colleagues.

A search for UNIX includes:

Database Systems: Structure of a Database File System for the UNIX Operating System 

Rochkind, M.J.

UNIX Time-Sharing System: The C Programming Language 

Ritchie, D.M.; Johnson, S.C.; Lesk, M.E.; Kernighan, B.W.

UNIX Time-Sharing System: The UNIX Shell 

Bourne, S.R.

UNIX Time-Sharing System: A Retrospective 

Ritchie, D.M.

UNIX Time-Sharing System: The Network Operations Center System 

 Cohen, H.; Kaufeld, J.C. Jr.

UNIX Time-Sharing System: Document Preparation 

Kernighan, B.W.; Lesk, M.E.; Ossanna, J.F. Jr.

The UNIX Time-Sharing System 

Ritchie, D.M.; Thompson, K.

UNIX Time-Sharing System: UNIX on a Microprocessor 

Lycklama, H.

Network Management Hint: Sort by IP Address

Network Management Hint: Sort by IP Address

How do you sort a host table by IP Address on a UNIX network management platform?

This has been a basic requirement, since the dawn of The Internet.

Let's look at an example /etc/hosts extract:

sun9999/user$ cat /etc/hosts

#
127.0.0.1       localhost
::1             localhost

201.14.24.17    BT-Site-1       # LasPalmas
203.16.54.112   QB-Site-1       # NorthSydney
201.14.24.21    DL-Site-1       # LasPalmas
202.135.192.97  QB-Site-1       # NorthSydney
203.16.54.18    PR-Site-1       # NorthSydney
201.14.24.28    DL-Site-2       # LasPalmas
203.16.54.22    KP-Site-1       # NorthSydney

Each octet in an IPv4 address is separated by a period "." and a simple POSIX sort will use the period as the field separator and then request a numeric sort on the first, second, third, and fourth octets.

sun9999/user$ sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n /etc/hosts

::1 localhost
#
127.0.0.1       localhost
201.14.24.17    BT-Site-1     # LasPalmas
201.14.24.21    DL-Site-1     # LasPalmas
201.14.24.28    DL-Site-2     # LasPalmas
202.135.192.97  QB-Site-1     # NorthSydney
203.16.54.18    PR-Site-1     # NorthSydney
203.16.54.22    KP-Site-1     # NorthSydney
203.16.54.112   QB-Site-1     # NorthSydney

What it you want to eliminate colon ":" separated IPv6 addresses, empty lines, and lines with comments? Just perform a little POSIX awk'ing after the sort.

sun9999/user$ sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n /etc/hosts | awk '$1!~/:/ && !/^$/ && !/^#/'

127.0.0.1       localhost
201.14.24.17    BT-Site-1       # LasPalmas
201.14.24.21    DL-Site-1       # LasPalmas
201.14.24.28    DL-Site-2       # LasPalmas
202.135.192.97  QB-Site-1       # NorthSydney
203.16.54.18    PR-Site-1       # NorthSydney
203.16.54.22    KP-Site-1       # NorthSydney
203.16.54.112   QB-Site-1       # NorthSydney

Want to eliminate all localhost entries? Add a localhost elimination pattern:

sun9999/user$ sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n /etc/hosts | awk '$1!~/:/ && !/^$/ && !/^#/ && !/localhost/'

201.14.24.17    BT-Site-1       # LasPalmas
201.14.24.21    DL-Site-1       # LasPalmas
201.14.24.28    DL-Site-2       # LasPalmas
202.135.192.97  QB-Site-1       # NorthSydney
203.16.54.18    PR-Site-1       # NorthSydney
203.16.54.22    KP-Site-1       # NorthSydney
203.16.54.112   QB-Site-1       # NorthSydney

Need to print only the host name, sorted by IP Address, for entries managed by NorthSydney NOC?

sun9999/user$ sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n /etc/hosts | 
awk '$1!~/:/ && !/^$/ && !/^#/ && !/localhost/ && /NorthSydney/ { 
print $2 }'
QB-Site-1
PR-Site-1
KP-Site-1
QB-Site-1

If you are not using POSIX commands to do standard network management work, you don't know what you are missing!

Cloud Migration: iPhone, iPodTouch, iPad

I considered a quick blog posting from Cloud Migration today:

I just keep thinking that the proliferation of iPads and tablets in the enterprise is leading us back to the path of thick client computing. Don't get me wrong, I love the iPad and believe it is a great device....for emails, surfing the web, playing music, playing games, and getting directions. However, it is as thick of a client device as you can get. In addition, just like 3270 screens were proprietary, they are a proprietary platform. It seems like just yesterday everyone was rushing to get off of client/server systems and move to thin client machines with browser based access.

Of course, I thought this was interesting, but this left me with a bunch of thoughts:

[courtesy: thenewiphone.info 2011 rumor]

> proliferation of iPads and tablets in the enterprise is leading us back
> to the path of thick client computing

That is a very interesting thought.

• UNIX is the firmware in the iPad, iPodTouch, iPhone
• UNIX does not make it thick, Sun Workstations were thin
• UNIX makes i* more Open.

There is virtually no customization on the client end,
so I don't quite think that i* are thick clients.


> just like 3270 screens were proprietary

iPhone's and iPad's were specifically called out, "tablets" suggest Windows, but Android is not quite Open...

• Does Android comply with POSIX? OpenFirmware?
• Is Android getting sued for using Java?

What are the thin alternatives?

• SunRay's used to be SPARC based, but no longer. SPARC was Open.
• SunRay's were never based upon Solaris. Solaris was Open.
• SunRay's used a proprietary firmware, not based upon OpenBoot.

The SunRay's are more of an ultra-thin form factor, with firmware
that will update automatically (much the same way that i* devices
will, except the i* devices prompt the user for a convenient time
to update with the ability to customize their firmware.)

Other thoughts about thin clients:

• I don't see SunRay's in i* or tablet form-factors.
• I don't see SunRay's being sold by TelCo providers,
  as basic utilities leveraging their network infrastructure.
• I don't see SunRay's clients provided by non-Oracle vendors

Don't get me wrong, I have 3 SunRay's on my desktop, this very moment,
running SPARC Solaris OpenLook desktops (CDE and JDS are way too heavy
and difficult to customize for real business usage.) All our users run
third-party apps off of an internal Solaris cloud that I built years ago.

Right now, the i* format factor is less expensive, easier to use,
and perceived as more open than other thin client technologies.

Honestly, there is no reason why clouds should not be built on SunRay's.


If clouds are not using SunRay's, then Oracle needs to figure out how to
fix it, and I will be the first one on-board to advocate migrating my
decade old private Solaris SPARC cloud providing 300 thin clients!

UNIX/Linux Vocabulary Building

The UNIX/Linux environment is a rich collaboration of tools, tricks, and jokes built by generations of users with widely varying levels of ability. While basic competence is achievable within a short period of time (i.e. "Just 5-10 years to learn the rules and only a couple lifetimes to master.") it's easy to become dependent on a few commands when other interesting or more suitable tools are readily available:

# find / | grep ifconfig
vs
$ whereis ifconfig


$ man ls
vs
$ pinfo ls

In this spirit I recommend the following links (not surprisingly, Dave's favorite AWK is listed in both).

Reddit thread:
Give Me That One Command You Wish You Knew Years Ago

Beware the spelling errors:
Advanced Unix Commands

Death of Two Titans: Jobs and Ritchie

The Mac Observer published last week:

U.S. President Barack Obama issued a statement on the passing of Steve Jobs Wednesday calling the Apple cofounder one of America’s greatest innovators. Mr. Jobs passed away sometime on Wednesday, an event that had lead to many luminaries offering statements and remembrances about the iconic executive. The president, an iPad user, also praised Mr. Jobs for having, “changed the way each of us sees the world.”

[Steve Jobs]

Steve Jobs was famous for the creation of Apple Computer with the Apple II and Macintosh, as well as popular consumer electronics like the iPod, iPhone, and iPad.

Electronista publishes today:

Just one week after the tech industry was hit with the news of Steve Jobs' death, computer scientist Dennis Ritchie​ has also passed. The unfortunate news was announced by friend and colleague Rob Pike​, who notes that Ritchie died at his home after a "long illness." The 70-year-old Harvard graduate and veteran of Lucent Technologies and Bell Labs​ was credited with authoring the C programming language.

Dennis Ritchie published the book "The C Programming Language" in 1978 with Brian Kernighan​, after the C Language was developed at Bell Labs. The language is often referred to as "K&R C". This language was also used in the development of the UNIX Operating System with Ken Thompson.


[President Clinton awarding National Medal of Technology in 1998 to Dennis Ritchie and Ken Thompson]

Nearly all modern operating systems are now based upon C. These three are the grandfathers of modern computers. Even Steve Job's Macintosh Operating System, MacOS X, is based on UNIX and C.


I could never give justice to the likes of Steve Jobs or Dennis Ritchie in this blog, but at least I can reference them. Unlike the license place, they "lived free and died - UNIX".

May God rest their souls, which longed for excellence and perfection.

In case you have not seen the article a trade journal - this is from Information Week:

Q&A: Ken Thompson, Creator Of Unix

Ken Thompson, creator of Unix speaks with Dr. Dobbs on a wide variety of topics.

By Andrew Binstock
June 27, 2011 12:00 AM

Unix's ubiquity was a surprise, says Thompson

The Japan Prize, one of the highest honors awarded for outstanding contribution to science and technology, was awarded jointly this year to Ken Thompson and Dennis Ritchie for the creation of Unix.

The prize is normally given to the recipients at a lavish banquet in Tokyo attended by the emperor. However, due to the April earthquake and tsunami, the prizes this year were distributed at the honorees' place of work. Ken Thompson's ceremony was held at Google headquarters, where he currently works, and afterward he spoke with Dr. Dobb's editor in chief Andrew Binstock on a wide variety of topics. Their discussion ranged from the development of Unix, beginning in the late 1960s, to the collaboration with Ritchie, to Thompson's current project, the open source programming language Go, which is designed to get the most performance out of today's multicore and networked computers.

On Developing Unix

...

On The Go Language

...

On Collaborating With Dennis Ritchie

...

[rest of article]