Monday, May 31, 2021

Sun SPARC Enterprise T5120 - USB Boot

Abstract

UNIX Systems Manufacturers originated their markets as workstations, during a time when they used 32 bit systems and the rest of the PC market was concentrating on 8 and 16 bit systems, and some CPU vendors like Intel use segmentation to keep their 16 bit software alive while struggling to move to 32 bit architectures. Some of the original servers were stacked workstations on a rack in a cabinet. The former high-powered video cards were merely ignored, as remote management needed command line interfaces. Engineering quickly determined that console access needed to be built into a new class of systems: rack mounted servers. These early servers offered boot functionality from Network and Disk. One such boot capability was from USB Disk..

Sun Enterprise T5120

The Sun Enterprise T5120 is a server with a second generation OpenSPARC processor. It comes with a Lights Out Management (LOM) capability referred to as Integrated Lights Out Management (ILOM.) The Advanced Lights Out Management (ALOM) shell may be it's default. Most remote systems management work can be done from the LOM. The system, when looking at the front of the chassis: the T5120 has 2x USB ports next to the DVD drive on the right and 2x USB ports located in the back left corner.

DVD Drive USB Ports

When a SanDisk USB Flash Sticks are plugged into the USB ports located to the right of the DVD drive, they can be seen at the OpenFirmware prompt, and can be selected into a copy-paste buffer, for easy use.

{0} ok show-disks
a) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/disk
b) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/disk
c) /pci@0/pci@0/pci@2/scsi@0/disk
d) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
e) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@1/disk
f) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk
g) /iscsi-hba/disk
q) NO SELECTION
Enter Selection, q to quit: d
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
for creating devalias mydev for /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
{0} ok

Note: the USB stick in position "d" (this lettered position may change as new USB sticks are plugged or unplugged) has it's device name copied into a "copy-paste" buffer by selecting "d"

Failed Boot from a USB Stick

It looks like this when one boots from a USB stick with no operating system & boot environment on it:

{0} ok boot ^Y
{0} ok boot /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
Boot device: /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk File and args:
The file just loaded does not appear to be executable.
{0} ok

Creating a USB Boot Stick

The USB port can be used to create boot environment that the chassis is compatible with or even not compatible with! For example, creating a Solaris 11.4 USB Boot Stick from Solaris 11.3 after inserting a SanDisk USB stick into the front port next to the DVD Drive:

T5120/root# echo | format -e | grep -i SanDisk
4. c7t0d0 <SanDisk'-Cruzer Fit-1.00 cyl 1945 alt 0 hd 255 sec 63>
5. c8t0d0 <SanDisk'-Cruzer Fit-1.00 cyl 1945 alt 0 hd 255 sec 63>

T5120/root# ls -al *usb
-rw-r--r--   1 dh127087 staff    1217341440 May 3 19:38 sol-11_4-text-sparc.usb

T5120/root# time dd bs=16k if=sol-11_4-text-sparc.usb of=/dev/rdsk/c7t0d0s2
74300+1 records in
74300+1 records out

real    8m57.25s
user    0m0.47s
sys     0m13.99s

T5120/root# echo "par\nprint\n" | format -e c7t0d0 | tail -14 | nawk '$NF!="0" && !/partition/'
Total disk cylinders available: 148 + 0 (reserved cylinders)

Part      Tag    Flag     Cylinders       Size            Blocks
0 unassigned    wm       0 - 147        1.13GB    (148/0/0) 2377620
2 unassigned    wm       0 - 147        1.13GB    (148/0/0) 2377620

T5120/root#

This USB stick can now be tested from, from OpenBoot Firmware

Test Boot Solaris 11.4

After shutting down the OS, while on the console port, attempt to boot from 11.4, which is too new:

T5120/root# cd / ; sync ; sync ; init 0

svc.startd: The system is coming down. Please wait.
svc.startd: 137 system services are now being stopped.
syncing file systems... done
Program terminated
ChassisSerialNumber BEL07492JB

SPARC Enterprise T5120, No Keyboard
Copyright (c) 1998, 2017, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.33.6.h, 16256 MB memory available, Serial #78384094.
Ethernet address 0:14:4f:ac:b:de, Host ID: 84ac0bde.

{0} ok show-disks
a) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/disk
b) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/disk
c) /pci@0/pci@0/pci@2/scsi@0/disk
d) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
e) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@1/disk
f) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk
g) /iscsi-hba/disk
q) NO SELECTION
Enter Selection, q to quit: d
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
for creating devalias mydev for /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk

{0} ok boot ^Y
{0} ok boot /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
Boot device: /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk File and args:

'cpu:SUNW,UltraSPARC-T2:SUNW,sun4v-cpu:sun4v' is not supported by this release of Solaris.
Program terminated
ChassisSerialNumber BEL07492JB

SPARC Enterprise T5120, No Keyboard
Copyright (c) 1998, 2017, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.33.6.h, 16256 MB memory available, Serial #78384094.
Ethernet address 0:14:4f:ac:b:de, Host ID: 84ac0bde.

{0} ok

Note: The OpenSPARC T2 processor is not supported by Oracle Solaris 11.4, but Sun Microsystems Solaris 10, OpenSolaris, and Oracle Solaris 11.0 - Solaris 11.3 are all supported on the chassis.

How to get to an OK prompt on a SPARC

[Logo of Sun Microsystems]

How to Get to an OK prompt on a SPARC

Abstract:

From the beginning of reasonable UNIX Workstation Time, the Workstation typically had a physical hardware layer, a firmware layer, and an OS layer. The Firmware layer used by UNIX workstations was Open-Sourced by Sun Microsystems as OpenBoot, given to the market for vendors like IBM, Apple, etc. The FORTH based environment was called OpenFirmware by the IEEE. One such feature is to boot the operating system directly, from a pre-selected disk, in a pre-selected order. When booting from a different boot environment, like a USB stick, this can be stopped. All of this can be achieved from the OpenBoot "OK" prompt.

[Sun Microsystems Type 5 Keyboard, courtesy Deskthority]

From a Physical Console

If you are lucky enough to have a real SUN Keyboard, Video Monitor, and Mouse (KVM), breaking out to OpenBoot is easily achieved by using the following key sequence "[L1] [A]". The "[L1]" key may be labeled "Stop" on some newer keyboards, but the position is the same, so the simultaneous key sequence would be "[Stop][A]".

{0} ok

From an ILOM Console

If one is not lucky enough to have a physical console, it can be accomplished from an ILOM Console.

T5120/user$ ssh root@SUNT4-2-ilom
Password:

Oracle(R) Integrated Lights Out Manager
Version 3.2.6.8 r128095
Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved.
Warning: HTTPS certificate is set to factory default.
Hostname: ORACLESP-1207BDY07F
->

Go Directly to OK from ILOM Console

If the system AutoBoot is not configured, the OK prompt will appear on a powered up chassis console

-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

{0} ok

Go to OK from ILOM Console & Booted OS

If the system AutoBoot is configured, the OS prompt will appear on a powered up chassis console

-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

sunt4-2 console login: root
password: ****

sunt4-2/root# cd / ; sync ; init 0

{0} ok

Go to OK from ILOM Console & Booted OS without root

If the system AutoBoot is configured, the OS prompt will appear on a powered up chassis console.
If there is no root password for the system and the intention is to perform a re-installation,
disable auto-boot from the ILOM and perform a reset.

-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

sunt4-2 console login: #.
Serial console stopped.

-> set /HOST/domain/control auto-boot=disable
Set 'auto-boot' to 'disable' [disabled]

-> show /HOST/domain/control

/HOST/domain/control
    Targets:

    Properties:
        auto-boot = disabled
        boot_guests = enabled

    Commands:
        cd
        reset
        set
        show

-> reset /HOST/domain/control
Are you sure you want to reset /HOST/domain/control (y/n)? y
Performing reset on /HOST/domain/control

-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

Serial console started. To stop, type #.

svc.startd: The system is down.
syncing file systems... done
NOTICE: Entering OpenBoot.
NOTICE: Fetching Guest MD from HV.
NOTICE: Starting additional cpus.
NOTICE: Initializing LDC services.
NOTICE: Probing PCI devices.
NOTICE: Finished PCI probing.

SPARC T4-2, No Keyboard
Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.38.16, 511.5000 GB memory available, Serial #99740538.
Ethernet address 0:21:28:f1:eb:7a, Host ID: 85f1eb7a.

{0} ok

Conclusions:

The SPARC Chassis, with various access methodologies, can achieve access to the OpenBoot / OpenFirmware prompt, for loading new operating systems, regardless of what was previously loaded on the chassis.

Monday, May 24, 2021

Oracle SPARC T4-4 - USB Boot & Install

[SPARC T4-4 Chassis Image, Courtesy Oracle]

Oracle SPARC T4-4 - USB Boot & Install

Abstract

Creating a USB Boot Stick from Solaris

The USB port can be used to create a Solaris 11.4 USB Boot Stick from Solaris, after inserting a SanDisk USB stick into the front port next to the DVD Drive:

T5120/root# echo | format -e | grep -i SanDisk
4. c7t0d0 <SanDisk'-Cruzer Fit-1.00 cyl 1945 alt 0 hd 255 sec 63>
5. c8t0d0 <SanDisk'-Cruzer Fit-1.00 cyl 1945 alt 0 hd 255 sec 63>

This USB stick can now be tested from, from OpenBoot Firmware

Oracle SPARC T4-4

The Oracle SPARC T4-4 is a server with a 4th generation OpenSPARC processor. The SPARC T4 processor was manufactured to the same process size as the SPARC T3 CPU processor, but the core was upgraded, so equivalent throughput could be reached with half as many cores, at the same processor speed. The T4-4 Chassis comes with a Lights Out Management (LOM) capability referred to as Integrated Lights Out Management (ILOM.) Most remote systems management work can be done from the LOM. The system, when looking at the front of the chassis: the T5120 has 2x USB ports next to the DVD drive on the right and 2x USB ports located in the back left corner.

Attaching to the ILOM

The ILOM can be attached to via TCP/IP, if previously configured, or over a serial port.

T5120/user$ ssh root@sun1234-ilom
Password:

Oracle(R) Integrated Lights Out Manager
Version 3.2.6.8 r128095
Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved.
Hostname: ORACLESP-1207BDY075

->

Insert USB Boot

The USB boot flash stick should be inserted into a powered down chassis, to boot from firmware.

-> show /system power_state actual_power_consumption

/System
    Properties:
        power_state = Off
        actual_power_consumption = 37 watts

Power Up Chassis

After the USB boot stick is inserted, the chassis should be powered up from the ILOM.

-> start /SYS
Are you sure you want to start /SYS (y/n)? y
Starting /SYS

After the chassis is powered on, once the power usage increases, attach to the console

-> show /system power_state actual_power_consumption

/System
    Properties:
        power_state = On
        actual_power_consumption = 1384 watts

-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

Serial console started. To stop, type #.

{0} ok

Note: If the chassis was recently started, the ability to access a prompt does not really occur until after the actual power consumption rises to an expected level.
Note: if the "ok" prompt does not appear, press [RETURN] or [ENTER] key

Select USB Port

{0} ok show-disks
a) /reboot-memory@0
b) /pci@700/pci@1/pci@0/pci@0/LSI,sas@0/disk
c) /pci@500/pci@1/pci@0/pci@1/pci@0/pci@3/SUNW,emlxs@0,1/fp@0,0/disk
d) /pci@500/pci@1/pci@0/pci@1/pci@0/pci@3/SUNW,emlxs@0/fp@0,0/disk
e) /pci@400/pci@2/pci@0/pci@1/pci@0/pci@3/SUNW,emlxs@0,1/fp@0,0/disk
f) /pci@400/pci@2/pci@0/pci@1/pci@0/pci@3/SUNW,emlxs@0/fp@0,0/disk
g) /pci@400/pci@1/pci@0/pci@8/pci@0/usb@0,2/hub@2/hub@3/storage@2/disk
h) /pci@400/pci@1/pci@0/pci@8/pci@0/usb@0,2/hub@2/storage@2/disk
i) /pci@400/pci@1/pci@0/pci@0/LSI,sas@0/disk
j) /iscsi-hba/disk
m) MORE SELECTIONS
q) NO SELECTION
Enter Selection, q to quit: h
/pci@400/pci@1/pci@0/pci@8/pci@0/usb@0,2/hub@2/storage@2/disk has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
for creating devalias mydev for /pci@400/pci@1/pci@0/pci@8/pci@0/usb@0,2/hub@2/storage@2/disk
{0} ok

Note: the USB stick in position "h" (this lettered position may change as new USB sticks are plugged or unplugged) has it's device name copied into a "copy-paste" buffer by selecting "h"

Boot Solaris 11.4 from USB

After shutting down the OS, while on the console port, attempt to boot from 11.4, which is too new:

{0} ok boot ^Y
{0} ok boot /pci@400/pci@1/pci@0/pci@8/pci@0/usb@0,2/hub@2/storage@2/disk
Boot device: /pci@400/pci@1/pci@0/pci@8/pci@0/usb@0,2/hub@2/storage@2/disk File and args:
/

Install Solaris 11.4

As the USB boot occurs, the Solaris 11.4 installer begins.

SunOS Release 5.11 Version 11.4.0.15.0 64-bit
Copyright (c) 1983, 2018, Oracle and/or its affiliates. All rights reserved.
Remounting root read/write
Probing for device nodes ...
Preparing image for use
Done mounting image
USB keyboard
1. Arabic                        15. Korean
2. Belgian                       16. Latin-American
3. Brazilian                     17. Norwegian
4. Canadian-Bilingual            18. Portuguese
5. Canadian-French               19. Russian
6. Danish                        20. Spanish
7. Dutch                         21. Swedish
8. Dvorak                        22. Swiss-French
9. Finnish                       23. Swiss-German
10. French                        24. Traditional-Chinese
11. German                        25. TurkishQ
12. Italian                       26. UK-English
13. Japanese-type6                27. US-English
14. Japanese
To select the keyboard layout, enter a number [default 27]: 27

1. Chinese - Simplified
2. Chinese - Traditional
3. English
4. French
5. German
6. Italian
7. Japanese
8. Korean
9. Portuguese - Brazil
10. Spanish
To select the language you wish to use, enter a number [default is 3]: 3

User selected: English
Configuring devices.
Hostname: solaris

Welcome to the Oracle Solaris installation menu

        1 Install Oracle Solaris
        2 Install Additional Drivers
        3 Shell
        4 Terminal type (currently xterm)
        5 Reboot

Please enter a number [1]: 1

                          Welcome to Oracle Solaris

   Thanks for choosing to install Oracle Solaris! This installer enables you
   to install the Oracle Solaris Operating System (OS) on SPARC or x86
   systems.

   The installation log will be at /system/volatile/install_log.

   How to navigate through this installer:
   - Use the function keys listed at the bottom of each screen to move from
     screen to screen and to perform other operations.
   - Use the up/down arrow keys to change the selection or to move between
     input fields.
   - If your keyboard does not have function keys, or they do not respond,
     press ESC; the legend at the bottom of the screen will change to show
     the ESC keys for navigation and other functions.

F2_Continue F6_Help F9_Quit

                              Discovery Selection

   Select discovery method for disks

    Local Disks            Discover local disks

    iSCSI                  Discover iSCSI LUNs

F2_Continue F3_Back F6_Help F9_Quit

                                     Disks

    Where should Oracle Solaris be installed?
    Minimum size: 4.1GB    Recommended minimum: 6.1GB

    Type     Size(GB) Boot Device
    ----------------------------------------------------------------------
    scsi        279.4   +   SYS/MB/HDD4                  HITACHI
    scsi        279.4       SYS/MB/HDD0                  HITACHI
    usb           1.1       c2t0d0                       SanDisk'       <

    The following slices were found on the disk.

      Slice        # Size(GB)             Slice        # Size(GB)
      ------------------------             ------------------------
      rpool        0     279.4             Unused       5       0.0
      Unused       1       0.0             Unused       6       0.0
      Unused       3       0.0             Unused       7       0.0
      Unused       4       0.0             backup       2     279.4

F2_Continue F3_Back F6_Help F9_Quit

                         Solaris Slices: 279.4GB scsi

   Oracle Solaris can be installed on the whole disk or a slice on the disk.

   The following slices were found on the disk.

     Slice        # Size(GB)             Slice        # Size(GB)
     ------------------------             ------------------------
     rpool        0     279.4             Unused       5       0.0
     Unused       1       0.0             Unused       6       0.0
     Unused       3       0.0             Unused       7       0.0
     Unused       4       0.0             backup       2     279.4

                              Use the whole disk
                            Use a slice on the disk

F2_Continue F3_Back F6_Help F9_Quit

                                System Identity

   Enter a name for this computer that identifies it on the network.
   It can contain letters, numbers, periods (.) and minus signs (-). The
   name must start and end with an alphanumeric character and must contain
   at least one non-digit character.

   Computer Name: solaris

F2_Continue F3_Back F6_Help F9_Quit

   Select how the network interface should be configured.

     DHCP              Allow DHCP to configure the interface

     Static            Configure the interface with a static IP address

F2_Continue F3_Back F6_Help F9_Quit

                              Time Zone: Regions

    Select the region that contains your time zone.

      Regions
      ----------------------------------------
      UTC/GMT
      Africa
      Americas
      Antarctica
      Asia
      Atlantic Ocean
      Australia
      Europe
      Indian Ocean
      Pacific Ocean

F2_Continue F3_Back F6_Help F9_Quit

                               Locale: Language

   Select the default language support and locale specific data format.
   These selections determine the language support, the default date and
   time, and other data formats.
   The language chosen automatically determines the available territories.

     Language
    ----------------------------------------
     No Default Language Support
     Chinese
     English
     French
     German
     Italian
     Japanese
     Korean
     Portuguese
     Spanish

F2_Continue F3_Back F6_Help F9_Quit

                               Locale: Territory

   Select the language territory

     Territory
    ----------------------------------------
     United States (en_US.ISO8859-1)
     United States (en_US.ISO8859-15)
     United States (en_US.ISO8859-15@euro)
     United States (en_US.UTF-8)

F2_Continue F3_Back F6_Help F9_Quit

                                 Date and Time

   Edit the date and time as necessary.
   Time shown is the system clock time in UTC and will be interpreted as
   such on installation.
   The time is in 24 hour format.

     Year:    2021 (YYYY)
     Month:     05 (1-12)
     Day:       22 (1-31)
     Hour:      05 (0-23)
     Minute:    46 (0-59)

F2_Continue F3_Back F6_Help F9_Quit

                                     Users

    Define a root password for the system and user account for yourself.

    System Root Password (required)

     Root password:    solar1s
     Confirm password: solar1s

    Create a user account (optional)

     Your real name:
     Username:
     User password:
     Confirm password:

F2_Continue F3_Back F6_Help F9_Quit

                            Support - Registration

     Provide your My Oracle Support credentials to be informed of
     security issues, enable Oracle Auto Service Requests.

     See http://www.oracle.com/goto/solarisautoreg for details.

       Email: anonymous@oracle.com
              Easier for you if you use your My Oracle Support email
              address/username.

       Please enter your password if you wish to receive security
       updates via My Oracle Support.

              My Oracle Support password:

F2_Continue F3_Back F6_Help F9_Quit

                           Installing Oracle Solaris

               Preparing for Installation

              [                       (4%)                       ]

F9_Quit

                             Installation Complete

   The installation of Oracle Solaris has completed successfully.

   Reboot to start the newly installed software or Quit if you wish to
   perform additional tasks before rebooting.

   The installation log is available at /system/volatile/install_log. After
   reboot it can be found at /var/log/install/install_log.

F4_View Log F7_Halt F8_Reboot F9_Quit

May 22 06:22:23 solaris reboot: initiated by root

NOTICE: Entering OpenBoot.
NOTICE: Fetching Guest MD from HV.
NOTICE: Starting additional cpus.
NOTICE: Initializing LDC services.
NOTICE: Probing PCI devices.

NOTICE: Finished PCI probing.

SPARC T4-4, No Keyboard
Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.38.16, 1023.5000 GB memory available, Serial #99743488.
Ethernet address 0:21:28:f1:f7:0, Host ID: 85f1f700.

Boot device: /pci@400/pci@1/pci@0/pci@0/LSI,sas@0/disk@w5000cca0252bf86d,0:a File and args:

Loading smf(7) service descriptions: 238/238
Booting to milestone "svc:/milestone/config:default".

Configuring devices.
Loading smf(7) service descriptions: 2/2
Booting to milestone "all".

Hostname: solaris
May 22 06:35:06 solaris sendmail[1502]: My unqualified host name (solaris) unknown; sleeping for retry
May 22 06:35:06 solaris sendmail[1507]: My unqualified host name (solaris) unknown; sleeping for retry

solaris console login:

UnConfigure / Configure

Solaris 10, offered an option to perform a "sys-unconfig", to restore an OS back to factory settings. In Solaris 11, this has been replaced with another option called "sysconfig configure -s" or "sysconfig configure -s --destructive" to destroy the initial user home account to also be destroyed.

This option is good if moving a chassis to a different location.

Monday, May 17, 2021

Oracle Database Pre-Requisites Package

Abstract:

The Oracle RDBMS Database has long been a tool on UNIX platforms. With the acquisition of Sun Microsystems, Oracle has drawn the Solaris operating system closer to it's orbit. One such change was to make a "pre-requisites" package in Solaris 11, to simplify installation of the RDBMS. There is a caution, if you already have standardized on user & group names for database usage.

Solaris Pre-Requisites

I was not going to write anything about this, but it appears that even other skilled engineers have run into problems with the automation of the Solaris Pre-Requisites package. I had seen a blog post from Alan, who graciously provided a solution when deploying hundreds of systems automatically.

sun9876/root# pkg install oracle-database-preinstall-19c

Alan struggled with ZFS Filesystems being created for new users by the pre-requisites script, before the user filesystems were mounted. This is not the only place where ZFS Filesystems being created for users are a problem, this author experienced several other conditions, one such condition kept a production system from coming up when the id's were scrambled by another such script.

This author filed a BUG to stop creating a ZFS File System for every user created. We have hundreds of users on some of our servers, so this ZFS feature was an inhibition to moving from Solaris 10 to 11. If you enable this feature in 11.4, you might be able to avoid the "avoid" that Alan had to use, if you don't mind the user & group id's being re-created.

Not all data centers deploy hundreds of Oracle DB's simultaneously. This author kept the pre-requisites script, but the default nature "blew away" our old user & group id's, adjusted ownership, and it was a nightmare. Procedures and scripts were built to undo everything that the pre-requisites package did with the id's & home dirctory, including returning the oracle user & dba groups back to their original ID's, correcting user & group ownership in /export/home, and then correcting the passwd, shadow, and group database. What a nightmare!

Conclusions

Honestly, it should be considered a BUG to rip out user & group names if those user & group names already exist, as a package overwrites them. After doing some DBA reading (this author is a systems guy first now a days) and do what the DBA's request of me, I found they missed a step which was to perform an "avoid" in order to avoid wiping out existing user & group id's for replacement, prior the preinstallation!

sun9876/root# pkg avoid oracle-database-os-configuration

There should NEVER be the need to perform an "avoid" from doing likely harm, but rather the package should be smart enough to realize id's exist and use them as defaults... and use an "avoid" clause, or something similar, so the "check" can be avoided and selectively allow the dangerous "blowing away & replacing" behavior.

Another piece of advise is to always perform a "dry-run" of an installation, before adding a package you are not aware of.

sun9876/root# pkg install -n oracle-database-preinstall-19c

The dry run will show details that need to be understood, before applying an actual package later.

Monday, May 10, 2021

Sun SPARC Enterprise T5120 - ILOM Introduction

Abstract

UNIX Systems Manufacturers originated their markets as workstations, during a time when they used 32 bit systems and the rest of the PC market was concentrating on 8 and 16 bit systems, and some CPU vendors like Intel use segmentation to keep their 16 bit software alive while struggling to move to 32 bit architectures. Some of the original servers were stacked workstations on a rack in a cabinet. The former high-powered video cards were merely ignored, as remote management needed command line interfaces. Engineering quickly determined that console access needed to be built into a new class of systems: rack mounted servers. These early servers offered traditional ALOM compatibility shell as well as newer ILOM shell. The ALOM compatibility shell is quite functional.

Sun Enterprise T5120 LOM

The Sun Enterprise T5120 is a server with a second generation OpenSPARC processor. It comes with a Lights Out Management (LOM) capability referred to as Integrated Lights Out Management (ILOM.) The Advanced Lights Out Management (ALOM) shell may be it's default. Most remote systems management work can be done from the LOM. Oracle has an ILOM 3.0 manual. There are also manuals formerly published by Sun Microsystems for OpenBoot 3.x and OpenBoot 4.x manuals.

ILOM: Logging In

A 9600 Baud Serial Cable can be added to the Console port, to provide immediate access.

SUNSP00144FAC0BE7 login: admin
Password:
Waiting for daemons to initialize...

Daemons ready

Oracle(R) Integrated Lights Out Manager
Version 3.0.12.4.y r77080
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

->

ILOM: Help Screen

The Console port, provides help screen, less reasonable from the ALOM:

-> help
The help command is used to view information about commands and targets

Usage: help [-o|-output terse|verbose] [<command>|legal|targets|<target>|<target> <property>]

Special characters used in the help command are
[]   encloses optional keywords or options
<>   encloses a description of the keyword
     (If <> is not present, an actual keyword is indicated)
|    indicates a choice of keywords or options

help <target>              displays description if this target and its properties
help <target> <property>   displays description of this property of this target
help targets               displays a list of targets
help legal                 displays the product legal notice

Commands are:
cd
create
delete
dump
exit
help
load
reset
set
show
start
stop
version

->

Note: ALOM functions & objects are flat; ILOM functions are consistent with hierarchical objects

ILOM: Setting the Date

The LOM Serial Port provides access to set the date, as it is likely not correct.:

-> show /SP/clock

/SP/clock
    Targets:

    Properties:
        datetime = Mon May 10 14:36:06 2021
        timezone = EDT (America/New_York)
        uptime = 7 days, 03:28:24
        usentpserver = disabled

    Commands:
        cd
        set
        show

-> set /SP/clock datetime="051014402021"
Set 'datetime' to '051014402021'

-> show /SP/clock datetime

/SP/clock
Properties:
datetime = Mon May 10 14:40:08 2021

ILOM: Show System Controller Network

The LOM can be enabled to perform DHCP, so access can be made over an ethernet cable.

-> show /SP/network

/SP/network
    Targets:
        test

    Properties:
        commitpending = (Cannot show property)
        dhcp_server_ip = 192.168.1.254
        ipaddress = 192.168.1.110
        ipdiscovery = dhcp
        ipgateway = 192.168.1.254
        ipnetmask = 255.255.255.0
        macaddress = 00:14:4F:AC:0B:E7
        pendingipaddress = 192.168.1.110
        pendingipdiscovery = dhcp
        pendingipgateway = 192.168.1.254
        pendingipnetmask = 255.255.255.0
        state = enabled

    Commands:
        cd
        set
        show

->

ILOM: Access via TCP/IP

The LOM can be accessed over TCP/IP, over an ethernet cable, via a terminal package like PuTTY.

login as: root
Using keyboard-interactive authentication.
Password:
Waiting for daemons to initialize...

Daemons ready

Oracle(R) Integrated Lights Out Manager
Version 3.0.12.4.y r77080
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

->

ILOM: Chassis Power On

The Chassis can be powered on from the ILOM prompt

-> show /SYS

/SYS
    Targets:
        SERVICE
        LOCATE
        ACT
        PS_FAULT
        TEMP_FAULT
        FAN_FAULT
        MB
        HDD0
        HDD1
        HDD2
        HDD3
        PDB
        SASBP
        DVD
        TTYA
        USBBD
        FANBD0
        FANBD1
        PS0
        PS1
        VPS

    Properties:
        type = Host System
        ipmi_name = /SYS
        keyswitch_state = Normal
        product_name = SPARC-Enterprise-T5120
        product_part_number = 602-3580-08
        product_serial_number = BEL07492JB
        product_manufacturer = Oracle Corporation
        fault_state = Faulted
        power_state = Off

    Commands:
        cd
        reset
        set
        show
        start
        stop

-> start /SYS
Are you sure you want to start /SYS (y/n)? y
Starting /SYS

-> show /SYS power_state

/SYS
Properties:
power_state = On

ILOM: Chassis Power Status

The Chassis power status can be shown from the LOM prompt

-> show /SYS power_state

/SYS
Properties:
power_state = On

ILOM: Gain Firmware/OS Console

The Chassis LOM prompt can also provide Console access to Firmware and OS Console

-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

Serial console started. To stop, type #.
0:0:0>
0:0:0>POST 4.33.6.h 2017/05/04 14:30
0:0:0>
0:0:0>Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
0:0:0>POST enabling CMP 0 threads: ffffffff.ffffffff
0:0:0>VBSC mode is: 00000000.00000001
0:0:0>VBSC level is: 00000000.00000001
0:0:0>VBSC selecting Normal mode, MAX Testing.
0:0:0>VBSC setting verbosity level 2
0:0:0>Basic Memory Tests....Done
0:0:0>Test Memory....Done
0:0:0>Setup POST Mailbox ....Done
0:0:0>Master CPU Tests Basic....Done
0:0:0>Init MMU.....
0:0:0>NCU Setup and PIU link train....Done
0:0:0>L2 Tests....Done
0:0:0>Extended CPU Tests....Done
0:0:0>Scrub Memory....Done
0:0:0>SPU CWQ Tests...Done
0:0:0>MAU Tests...Done
0:0:0>Network Interface Unit Port 0 Tests ..Done
0:0:0>Network Interface Unit Port 1 Tests ..Done
0:0:0>Functional CPU Tests....Done
0:0:0>Extended Memory Tests....Done
2021-05-04 13:54:19.122 0:0:0>INFO:
2021-05-04 13:54:19.174 0:0:0> POST Passed all devices.
2021-05-04 13:54:19.229 0:0:0>POST: Return to VBSC.
2021-05-04 13:54:19.283 0:0:0>Master set ACK for vbsc runpost command and spin...
ChassisSerialNumber BEL07492JB

SPARC Enterprise T5120, No Keyboard
Copyright (c) 1998, 2017, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.33.6.h, 16256 MB memory available, Serial #78384094.
Ethernet address 0:14:4f:ac:b:de, Host ID: 84ac0bde.

{0} ok
Note: the "ok" prompt provides access to OpenFirmware

ILOM: OpenFirmware: Show Devices

The Chassis LOM Firmware prompt allows access all devices available to an operating system

{0} ok show-devs
/ebus@c0
/pci-performance-counters@0
/niu@80
/pci@0
/cpu@3f
/cpu@3e
/cpu@3d
/cpu@3c
/cpu@3b
/cpu@3a
/cpu@39
/cpu@38
/cpu@37
/cpu@36
/cpu@35
/cpu@34
/cpu@33
/cpu@32
/cpu@31
/cpu@30
/cpu@2f
/cpu@2e
/cpu@2d
/cpu@2c
/cpu@2b
/cpu@2a
/cpu@29
/cpu@28
/cpu@27
/cpu@26
/cpu@25
/cpu@24
/cpu@23
/cpu@22
/cpu@21
/cpu@20
/cpu@1f
/cpu@1e
/cpu@1d
/cpu@1c
/cpu@1b
/cpu@1a
/cpu@19
/cpu@18
/cpu@17
/cpu@16
/cpu@15
/cpu@14
/cpu@13
/cpu@12
/cpu@11
/cpu@10
/cpu@f
/cpu@e
/cpu@d
/cpu@c
/cpu@b
/cpu@a
/cpu@9
/cpu@8
/cpu@7
/cpu@6
/cpu@5
/cpu@4
/cpu@3
/cpu@2
/cpu@1
/cpu@0
/virtual-devices@100
/iscsi-hba
/virtual-memory
/memory@m0,8000000
/aliases
/options
/openprom
/chosen
/packages
/ebus@c0/serial@0,ca0000
/pci@0/pci@0
/pci@0/pci@0/pci@9
/pci@0/pci@0/pci@8
/pci@0/pci@0/pci@2
/pci@0/pci@0/pci@1
/pci@0/pci@0/pci@8/pci@0
/pci@0/pci@0/pci@8/pci@0/pci@a
/pci@0/pci@0/pci@8/pci@0/pci@9
/pci@0/pci@0/pci@8/pci@0/pci@8
/pci@0/pci@0/pci@8/pci@0/pci@2
/pci@0/pci@0/pci@8/pci@0/pci@1
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/tape
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/disk
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0/fp@0,0
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/tape
/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/disk
/pci@0/pci@0/pci@2/scsi@0
/pci@0/pci@0/pci@2/scsi@0/disk
/pci@0/pci@0/pci@2/scsi@0/tape
/pci@0/pci@0/pci@1/pci@0
/pci@0/pci@0/pci@1/pci@0/pci@3
/pci@0/pci@0/pci@1/pci@0/pci@2
/pci@0/pci@0/pci@1/pci@0/pci@1
/pci@0/pci@0/pci@1/pci@0/pci@3/network@0,1
/pci@0/pci@0/pci@1/pci@0/pci@3/network@0
/pci@0/pci@0/pci@1/pci@0/pci@2/network@0,1
/pci@0/pci@0/pci@1/pci@0/pci@2/network@0
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,1
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3/disk
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1/disk
/virtual-devices@100/rtc@5
/virtual-devices@100/console@1
/virtual-devices@100/random-number-generator@e
/virtual-devices@100/ncp@6
/virtual-devices@100/n2cp@7
/virtual-devices@100/channel-devices@200
/virtual-devices@100/tpm@f
/virtual-devices@100/flashprom@0
/virtual-devices@100/channel-devices@200/virtual-domain-service@0
/virtual-devices@100/channel-devices@200/virtual-channel-client@1
/virtual-devices@100/channel-devices@200/virtual-channel@0
/virtual-devices@100/channel-devices@200/virtual-channel-client@2
/virtual-devices@100/channel-devices@200/virtual-channel@3
/iscsi-hba/disk
/openprom/client-services
/packages/obp-tftp
/packages/kbd-translator
/packages/SUNW,asr
/packages/dropins
/packages/terminal-emulator
/packages/disk-label
/packages/deblocker
/packages/SUNW,builtin-drivers

{0} ok

ILOM: OpenFirmware: Show Disks

The Chassis LOM Firmware prompt allows access the devices hosting the operating system

{0} ok

ILOM: OpenFirmware: Device Aliases

The Chassis LOM Firmware prompt provides access to common alias names for long device names.

{0} ok devalias
backup           /pci@0/pci@0/pci@2/scsi@0/disk@0
primary          /pci@0/pci@0/pci@2/scsi@0/disk@2
ttya             /ebus@c0/serial@0,ca0000
nvram            /virtual-devices/nvram@3
net3             /pci@0/pci@0/pci@1/pci@0/pci@3/network@0,1
net2             /pci@0/pci@0/pci@1/pci@0/pci@3/network@0
net1             /pci@0/pci@0/pci@1/pci@0/pci@2/network@0,1
net0             /pci@0/pci@0/pci@1/pci@0/pci@2/network@0
net              /pci@0/pci@0/pci@1/pci@0/pci@2/network@0
cdrom            /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk@0:f
disk3            /pci@0/pci@0/pci@2/scsi@0/disk@3
disk2            /pci@0/pci@0/pci@2/scsi@0/disk@2
disk1            /pci@0/pci@0/pci@2/scsi@0/disk@1
disk0            /pci@0/pci@0/pci@2/scsi@0/disk@0
disk             /pci@0/pci@0/pci@2/scsi@0/disk@0
scsi             /pci@0/pci@0/pci@2/scsi@0
virtual-console /virtual-devices/console@1
name             aliases

{0} ok

ILOM: OpenFirmware: Default Environment

The Chassis LOM Firmware prompt provides basic default values, which can be changed.

{0} ok printenv
Variable Name           Value                          Default Value
ttya-rts-dtr-off        false                          false
ttya-ignore-cd          true                           true
keyboard-layout         US-English
reboot-command
security-mode           none                           No default
security-password                                      No default
security-#badlogins     0                              No default
verbosity               min                            min
pci-mem64?              true                           true
diag-switch?            false                          false
local-mac-address?      true                           true
fcode-debug?            false                          false
scsi-initiator-id       7                              7
oem-logo                                               No default
oem-logo?               false                          false
oem-banner                                             No default
oem-banner?             false                          false
ansi-terminal?          true                           true
screen-#columns         80                             80
screen-#rows            34                             34
ttya-mode               9600,8,n,1,-                   9600,8,n,1,-
output-device           virtual-console                virtual-console
input-device            virtual-console                virtual-console
auto-boot-on-error?     false                          false
load-base               16384                          16384
auto-boot?              false                          true
network-boot-arguments
boot-command            boot                           boot
boot-file
boot-device             /pci@0/pci@0/pci@2/scsi@ ...   disk net
multipath-boot?         false                          false
boot-device-index       0                              0
use-nvramrc?            true                           false
nvramrc                 devalias primary /pci@0/ ...
error-reset-recovery    boot                           boot

{0} ok printenv boot-device
boot-device = /pci@0/pci@0/pci@2/scsi@0/disk@2,0:a /pci@0/pci@0/pci@2/scsi@0/disk@0,0:a

{0} ok

Note: The system is set to not automatically boot, but there is a default boot disk defined.

ILOM: OpenFirmware: Boot DVD OS Installer

The Chassis LOM OpenFirmware prompt can initiate an OS boot from DVD to install an OS.

{0} ok boot cdrom
Boot device: /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk@0:f File and args:
|

...

ILOM: OpenFirmware: Escape from OpenFirmware to LOM

The Chassis OpenFirmware can be exited with the escape sequence "#." to return to the ALOM.

{0} ok #.
Serial console stopped.

ILOM: Chassis Power Off

The Chassis can be powered off from the LOM prompt (This may take a few minutes)

-> stop /SYS
Are you sure you want to power off the system [y/n]? y

-> show /SYS power_state

/SYS
Properties:
power_state = Off
->

ILOM: Logging Out

the user is done with the Serial Port access to the LOM, they can log out to close the session.

-> exit

Conclusions

OpenSPARC Systems are very viable platforms, offering tremendous Lights Out Management capabilities. The most recent SPARC Systems have been the fastest platforms in the world for nearly 4 years, so there is a tremendous growth potential for migrating from these older Open Source SPARC platforms. Being based upon Open Hardware, Open Firmware, and Open Hardware - new hardware vendors can also create their own newer generation platforms, to meet their own requirements if the Light Out Management capabilities are not a requirement.

Reference Material

Monday, May 31, 2021

Sun SPARC Enterprise T5120 - USB Boot

Sun SPARC Enterprise T5120 - USB Boot

Abstract

Sun Enterprise T5120

DVD Drive USB Ports

Failed Boot from a USB Stick

Creating a USB Boot Stick

Test Boot Solaris 11.4

How to get to an OK prompt on a SPARC

How to Get to an OK prompt on a SPARC

Abstract:

From a Physical Console

From an ILOM Console

Go Directly to OK from ILOM Console

Go to OK from ILOM Console & Booted OS

Go to OK from ILOM Console & Booted OS without root

Conclusions:

Monday, May 24, 2021

Oracle SPARC T4-4 - USB Boot & Install

Oracle SPARC T4-4 - USB Boot & Install

Abstract

Creating a USB Boot Stick from Solaris

Oracle SPARC T4-4

Attaching to the ILOM

Insert USB Boot

Power Up Chassis

Select USB Port

Boot Solaris 11.4 from USB

Install Solaris 11.4

UnConfigure / Configure

Monday, May 17, 2021

Oracle Database Pre-Requisites Package

Oracle Database Pre-Requisites Package

Abstract:

Solaris Pre-Requisites

Conclusions

Monday, May 10, 2021

Sun SPARC Enterprise T5120 - ILOM Introduction

Sun SPARC Enterprise T5120 - ILOM Introduction

Abstract

Sun Enterprise T5120 LOM

ILOM: Logging In

ILOM: Help Screen

ILOM: Setting the Date

ILOM: Show System Controller Network

ILOM: Access via TCP/IP

ILOM: Chassis Power On

ILOM: Chassis Power Status

ILOM: Gain Firmware/OS Console

ILOM: OpenFirmware: Show Devices

ILOM: OpenFirmware: Show Disks

ILOM: OpenFirmware: Device Aliases

ILOM: OpenFirmware: Default Environment

ILOM: OpenFirmware: Boot DVD OS Installer

ILOM: OpenFirmware: Escape from OpenFirmware to LOM

ILOM: Chassis Power Off

ILOM: Logging Out

Conclusions