Showing posts with label Oracle. Show all posts
Showing posts with label Oracle. Show all posts

Wednesday, November 23, 2022

OpsCenter 12c 12.4, Patches, and Solaris 11.4 SRU 48

 

OpsCenter 12c 12.4, Patches, and Solaris 11.4 SRU 48

Abstract

Sun Microsystems purchased a company, which performed data center management, across multiple platforms, and then decided to merge it into Solaris. The Sun Connect product was born, to help deliver patches more effectively. The N1 Compute Initiative was born, to treat all systems in the data center as a single entity. OpsCenter was made available for all customers, to do local provisioning, patch, health, and reporting. Oracle purchased Sun Microsystems and had their own management framework called Oracle Enterprise Manager, but it was always short on handling hardware through provisioning hardware & management of the ILOM hardware [without an OS]. Oracle refers to OpsCenter as Oracle Enterprise Manager OpsCenter. 

Recent History

Oracle had been updating OpC pretty aggressively in 2022, this article talks about the path to accomplish this. The first set of updates was associated with Log4J vulnerability in December 2021, but introduction of Oracle Solaris 11.4 SRU 48 on agents actually caused a disconnection to the OpsCenter server, and some aspect of this procedure must be followed in order to restore connectivity to OpsCenter from managed Solaris 11.4 servers newer than 11.4.48.

 

OpsCenter Information Center

One of the most important sections to review is the OpsCenter Information Center, within Oracle's Support Network, for understanding what OpsCenter is and what updates are occurring.

Information Center: Overview of Enterprise Manager Ops Center (Doc ID 1507780.2)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=1507780.2

As of the writing of this article, there have been several CPU (Critical Patch Updates),
stemming from a Log4J vulnerability discovered in the industry in December 2021. 

This is a good place to start, regarding the latest news on OpsCenter.

OpsCenter 12.4 Release

Oracle upgraded OpsCenter to the 12.4 release in 2019. 

Release Announcement - Oracle Enterprise Manager Ops Center 12c Release 4 (12.4.0.0.0) ( Doc ID 2532906.1 ) April 2019
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2532906.1

The documentation is readily available to everyone, without My Oracle Support (MOS):

Oracle Enterprise Manager Ops Center 12c Release 4 (12.4)
https://docs.oracle.com/cd/ops-center-12.4/index.htm


Base Operating System

Ops Center 12c Release 12.4 is supported on Solaris 11.3 as well as Solaris 11.4, making it a well suited management tool which can be used on nearly any piece of Solaris SPARC Hardware.
 
Most installations will be on newer hardware, with security updates available for Solaris 11.4. Solaris 11.4, as of the time of this writing, is on SRU 50... this is the 50'th month after original release of 11.4!

To avoid installing a buggy Solaris 11.4.0 & applying a half-decade of patches to Solaris 11.4.50, Oracle released free for private use Solaris 11.4 CBE or Common Build Environment.
 
As of the time of this writing, this starts at Solaris 11.4 CBE starts at SRU 42, but OpsCenter will need to be upgraded to the support repository in order to get the required Perl XML parser.

OpsCenter 12c 12.4 Pre-Requisites

There are OpsCenter bugs which require workarounds, for installation on later releases of Solaris.

There is a known BUG (32548385) with OpsCenter, introduced by Solaris 11.4.30.
The python 'mediator' in Solaris 11.4 SRU 30 is set to 3.7 instead of 2.7. Ops Center requires 2.7.
Ops Center Will Not Start After Upgrading to Solaris 11.4.3- SRU 30 - Svc:/application/scn/ajaxterm:default is Restarting Too Quickly (Doc ID 2760685.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2760685.1

This mediator pre-requisite must be present for OpsCenter to start up on Solaris 11.4.30 and later.

There is a known BUG (33622838) with OpsCenter, introduced by Solaris 11.4.39.
Older release of Perl 5.26 in Solaris 11.4 SRU 30 is removed. OpsCenter EC requires Perl 5.2.2.
Ops Center 12.4 upgrades to Solaris 11.4 SRU 39 on an EC will fail (Doc ID 2826475.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2826475.1

This perl release pre-requisite must be present for OpsCenter to install on Solaris 11.4.39 and later.

OpsCenter 12.4 Installation

OpsCenter should be installed or upgraded to its most recent base version.

A basic installation with a Single Enterprise Controller is readily available:
https://docs.oracle.com/cd/ops-center-12.4/doc.1240/e59965/GUID-0DE73AE5-1B0B-4403-890A-8F632AD30131.htm#OPCSO525

After upgrade or installation, patches should be applied.

[Byzantine Mosaic: Jesus Christ Pantocrator, Courtesy Ricard MN Photography]

OpsCenter 12.4 Critical Patch Updates

Normally, Critical Patch Updates are cumulative, but this ceased to be the case after April 2022.
A circuitous path to follow, to deal with bureaucracy, was un-affectionately referred to as Byzantine.
This is where our Byzantine journey begins!

OpsCenter 12.4 April 2022 Critical Patch Update

The April 2022 CPU resolved a variety of issues, including Log4J.
(The January 2022 release, with Log4J patches, is also bundled in the April 2022 CPU.)

Ops Center 12.4 companion document for the April 2022 CPU (Doc ID 2865470.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2865470.1

Non-intuitively, this refers the user to another document, that says EM-only, but includes OpsCenter:

Critical Patch Update (CPU) Program Apr 2022 Patch Availability Document (EM-only) (Doc ID 2844807.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2844807.1

A "Bundle Patch" was created, which must be downloaded, and applied according to the instructions:

OPSS BUNDLE PATCH 12.2.1.4.210418 Patch 32784652 or later
https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?patchId=32784652

These are the major bugs which have been resolved:

Bug 33601961 - Ops Center 12.4: CVE-2021-40438 Apache HTTPD server
Bug 33490456 - CVE-2021-2351: UPDATE THE C CLIENT LIBRARY FOR NNE VULNERABILITY
Bug 33735042 - CVE-2021-44832: APACHE LOG4J UPDATE TO 2.3.2, 2.12.4, OR 2.17.1

This must be applied only on an installation of OpsCenter 12.4

OpsCenter 12.4 July 2022 Critical Patch Update

The July 2022 CPU resolved a variety of issues...

Ops Center 12.4 companion document for the July 2022 CPU (Doc ID 2885006.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2885006.1

Non-intuitively, this refers the user to another document, that says EM-only, but includes OpsCenter:

Critical Patch Update (CPU) Program Jul 2022 Patch Availability Document (EM-only) (Doc ID 2867874.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2867874.1&_afrWindowMode=0&_adf.ctrl-state=1b5ay5nont_123#babfaaai

 A "Bundle Patch" was created, which must be downloaded, and applied according to the instructions:

Ops Center UCE patches for Jul CPU 2022 Patch 34332927 or later
https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?patchId=34332927

These are the major bugs which have been resolved:

Bug 34259326 - Ops Center 12.4: CVE-2022-22720 in Apache 2.4.52
Bug 34259352 - Ops Center 12.4: CVE-2022-22721 in Apache 2.4.52
Bug 34269953 - Ops Center 12.4: Upgrade OpenSSL to 1.1.1o

This must be only applied after the April release.

OpsCenter 12.4 October 2022 Critical Patch Update

The October 2022 CPU resolved a variety of issue...

Ops Center 12.4 companion document for the Oct 2022 CPU (Doc ID 2904332.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2904332.1

Non-intuitively, this refers the user to another document, that says EM-only, but includes OpsCenter:

Critical Patch Update (CPU) Program Oct 2022 Patch Availability Document (EM-only) (Doc ID 2888514.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=298046759019691&id=2888514.1&_adf.ctrl-state=1b5ay5nont_659

A "Bundle Patch" was created, which must be downloaded, and applied according to the instructions:

Ops Center UI/Other patches for Oct CPU 2022 Patch 34611523 or later
https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?patchId=34611523

These are the major bugs which have been resolved:

Bug 33952830  CVE-2021-23450: DOJO UPDATE TO AT LEAST 1.17.0

This must be only applied after the April and July releases.

OS Upgrade to Solaris 11.4 SRU 48

This may seem counter intuitive, but there is a bug in SRU 48, which disconnects OpsCenter agent from the OpsCenter Proxy Controller, which reflects in a down agent in OpsCenter Enterprise Controller... and before you can fix this bug, the operating system must be upgraded to crash the agent, then the fix can be applied.

Fixing the OpsCenter OS Agent

With application of Oracle Solaris 11.4 SRU 48, the agent fails to connect to the management station.

A good article on the topic is:

Ops Center 12.4: CDOM Agents fail to start after a Solaris upgrade to 11.4 SRU 48 ( Doc ID 2892465.1 )
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2892465.1

At the root cause, there are a variety of bugs identified with SRU48.

Bug 34525568 : OpsCenter 12.4 CDOM Agents fail to start on Solaris SRU11.4.48 due to XMPP
Bug 34560282 : Ops Center Agent won't start after upgrade to 11.4SRU48 with S7 having Global zone
Bug 33876279 Local connections should skip TLS, SASL handshake

The patch to correct this problem, with SRU 48+, can only be acquired from the OpsCenter team, and is not generally available through the Oracle Patch Management system

A Service Request must be filed, specifically asking for the patch:

Patch 34525568


Posted by at No comments:
Labels: , , , , , , , , , , , , ,

Monday, October 4, 2021

Differences Between Solaris 10 and Solaris 11.4

 

Differences Between Solaris 10 and Solaris 11.4

Abstract:

Sun Microsystems used to migrate between operating systems fairly regularly. A new trend had come to the software development community referred to as Continuous Delivery. Oracle purchased Sun Microsystems. Solaris 10 acquired many new features, the life expectancy was extended significantly, and Solaris 11 was released mid-way through the significantly lengthened support cycle. Instead of releasing Solaris 12, Oracle made the executive decision to roll all features of Solaris 12 into Solaris 11.4.

Solaris 11.4 aka Solaris 12

What are some of the differences between Solaris 10 and Solaris 11.4?

Oracle published a document summarizing the differences, with links to major documents.

Key Differences between Oracle Solaris 10 and Oracle Solaris 11

Upgrading from Oracle Solaris 10 to Oracle Solaris 11 requires a fresh installation of Oracle Solaris 11.

Tools to help you make the transition include the following:

  • Oracle Solaris 10 branded zones. Migrate Oracle Solaris 10 instances to Oracle Solaris 10 zones on Oracle Solaris 11 systems.

  • ZFS shadow migration. Migrate UFS data from an existing file system, either local or NFS, to a new local ZFS file system. Do not mix UFS directories and ZFS file systems in the same file system hierarchy.

    You can also remotely mount UFS file systems from an Oracle Solaris 10 system onto an Oracle Solaris 11 system, or use the ufsrestore command on an Oracle Solaris 10 system to restore UFS data (ufsdump) into an Oracle Solaris 11 ZFS file system.

  • ZFS pool import. Export and disconnect storage devices that contain ZFS storage pools on your Oracle Solaris 10 systems and then import them into your Oracle Solaris 11 systems.

  • NFS file sharing. Share files from an Oracle Solaris 10 system to an Oracle Solaris 11 system. Do not mix NFS legacy shared ZFS file systems and ZFS NFS shared file systems. Use only ZFS NFS shared file systems.

For the main Oracle Solaris documentation, see Oracle Solaris Documentation. For additional documentation and examples, select a technology on the Oracle Solaris 11 Technology Spotlights page.

Applications that run on Oracle Solaris 10 should also run on Oracle Solaris 11 if they use only public Oracle Solaris interfaces. Oracle Solaris Preflight Applications Checker 11.3 can determine the Oracle Solaris 11 readiness of an application by analyzing the working application on Oracle Solaris 10. A successful check with this tool strongly indicates that you can run the application without modification on Oracle Solaris 11.

Versions of FOSS and other software are updated. In some cases, a system can have more than one version of a command or tool simultaneously installed. If your application depends on a particular version, use the full path to the executable rather than depend on a link.

See End of Feature Notices for Oracle Solaris 11 for lists of commands and tools that are no longer available in Oracle Solaris 11. In most cases, Oracle Solaris 11 provides alternative commands and tools. The list also includes hardware that does not support newer Oracle Solaris 11 versions.

A graphical desktop is not included by default with some system installations. If you want a graphical desktop, install the group/system/solaris-desktop IPS package.

Installation and Upgrade Changes

The following are key changes from Oracle Solaris 10 to Oracle Solaris 11:

  • Installation and upgrade:

    • Instead of JumpStart, use Automated Installer.

    • Instead of Live Upgrade, use the text installer or Image Packaging System (IPS) pkg commands.

    • Software packages are delivered in package repositories, similar to Linux package repositories.

  • Archive and recovery: Instead of Flash Archives, use Unified Archives.

  • System services: More system configuration is done by setting Service Management Facility (SMF) service property values and not by directly editing configuration files. Look for comments in the configuration files and see the documentation for that configuration.

  • root user: By default, root is a role, not a user. Instead of doing privileged tasks as root, create and assign roles targeted to each set of related tasks.

  • Shell: The default shell for the root user is ksh. The default shell for other users is bash. Default user PATH also has changed.

Changes in How to Configure Oracle Solaris Features

More configuration is provided by partial configuration files in the /etc/system.d directory, where customer-specific system configuration files should also be stored. Routinely editing /etc/system should be avoided. In some cases, the partial configuration file is created by an SMF service using service property values that you provide.

For network configuration, Oracle Solaris 11 assigns generic names to each datalink on a system by using the net0, net1, netN naming convention. Configuration is also managed through SMF service property values rather than by directly editing configuration files. In addition, new commands for setting up datalinks and IP interfaces have been introduced to replace the commonly used commands in Oracle Solaris 10, such as ifconfig.

Networking in Oracle Solaris 11 has advanced to provide better network performance, efficient network resource management, higher network availability, and new technologies such as in the area of network virtualization. See the documentation in Administering Oracle Solaris Networks and Administering Network Services in Oracle Solaris.

Changes in User Environment

  • Default login and other shell changes - In Oracle Solaris 11, /bin/sh is the Korn shell (ksh93), and the default interactive shell is the Bourne-again (bash) shell. When used as a login shell, bash retrieves configuration information from the first instance of .bash_profile, .bash_login, or .profile file.

    • The legacy Bourne shell is available as /usr/sunos/bin/sh.

    • The legacy ksh88 is available as /usr/sunos/bin/ksh from the shell/ksh88 package.

    • Korn shell compatibility information is available in /usr/share/doc/ksh/COMPATIBILITY.

  • Default user path and PATH environment variable – The default user path is /usr/bin. The default path for the root role is /usr/bin:/usr/sbin. The default PATH environment variable for bash is /usr/bin:/usr/sbin

For more details about user environment in Oracle Solaris 11.4, see About the User Work Environment in Managing User Accounts and User Environments in Oracle Solaris 11.3.

Changes in Security

Security in Oracle Solaris 11 supports industry standards more closely. For an overview of security in Oracle Solaris 11, see Security: An Oracle Solaris Differentiator.

Other enhancements increase hardening, add compliance functionality, and enable remote administration of security:

Posted by at No comments:
Labels: , , , , , ,

Monday, May 31, 2021

Sun SPARC Enterprise T5120 - USB Boot

 

Sun SPARC Enterprise T5120 - USB Boot

Abstract

UNIX Systems Manufacturers originated their markets as workstations, during a time when they used 32 bit systems and the rest of the PC market was concentrating on 8 and 16 bit systems, and some CPU vendors like Intel use segmentation to keep their 16 bit software alive while struggling to move to 32 bit architectures. Some of the original servers were stacked workstations on a rack in a cabinet. The former high-powered video cards were merely ignored, as remote management needed command line interfaces. Engineering quickly determined that console access needed to be built into a new class of systems: rack mounted servers. These early servers offered boot functionality from Network and Disk. One such boot capability was from USB Disk..


Sun Enterprise T5120

The Sun Enterprise T5120 is a server with a second generation OpenSPARC processor. It comes with a Lights Out Management (LOM) capability referred to as Integrated Lights Out Management (ILOM.) The Advanced Lights Out Management (ALOM) shell may be it's default. Most remote systems management work can be done from the LOM. The system, when looking at the front of the chassis: the T5120 has 2x USB ports next to the DVD drive on the right and 2x USB ports located in the back left corner.

DVD Drive USB Ports

When a SanDisk USB Flash Sticks are plugged into the USB ports located to the right of the DVD drive, they can be seen at the OpenFirmware prompt, and can be selected into a copy-paste buffer, for easy use.

{0} ok show-disks
a) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/disk
b) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/disk
c) /pci@0/pci@0/pci@2/scsi@0/disk
d) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
e) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@1/disk
f) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk
g) /iscsi-hba/disk
q) NO SELECTION
Enter Selection, q to quit: d
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
         for creating devalias mydev for /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
{0} ok 

Note: the USB stick in position "d" (this lettered position may change as new USB sticks are plugged or unplugged) has it's device name copied into a "copy-paste" buffer by selecting "d"

Failed Boot from a USB Stick

It looks like this when one boots from a USB stick with no operating system & boot environment on it:

{0} ok boot ^Y
{0} ok boot /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
Boot device: /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk  File and args:
The file just loaded does not appear to be executable.
{0} ok

Creating a USB Boot Stick

The USB port can be used to create boot environment that the chassis is compatible with or even not compatible with! For example, creating a Solaris 11.4 USB Boot Stick from Solaris 11.3 after inserting a SanDisk USB stick into the front port next to the DVD Drive:

T5120/root# echo | format -e | grep -i SanDisk
       4. c7t0d0 <SanDisk'-Cruzer Fit-1.00 cyl 1945 alt 0 hd 255 sec 63>
       5. c8t0d0 <SanDisk'-Cruzer Fit-1.00 cyl 1945 alt 0 hd 255 sec 63>

T5120/root# ls -al *usb
-rw-r--r--   1 dh127087 staff    1217341440 May  3 19:38 sol-11_4-text-sparc.usb

T5120/root# time dd bs=16k if=sol-11_4-text-sparc.usb of=/dev/rdsk/c7t0d0s2
74300+1 records in
74300+1 records out

real    8m57.25s
user    0m0.47s
sys     0m13.99s

T5120/root# echo "par\nprint\n" | format -e c7t0d0 | tail -14 | nawk '$NF!="0" && !/partition/'
Total disk cylinders available: 148 + 0 (reserved cylinders)

Part      Tag    Flag     Cylinders       Size            Blocks
  0 unassigned    wm       0 - 147        1.13GB    (148/0/0) 2377620
  2 unassigned    wm       0 - 147        1.13GB    (148/0/0) 2377620

T5120/root#

This USB stick can now be tested from, from OpenBoot Firmware

Test Boot Solaris 11.4

After shutting down the OS, while on the console port, attempt to boot from 11.4, which is too new:

T5120/root# cd / ; sync ; sync ; init 0
svc.startd: The system is coming down.  Please wait.
svc.startd: 137 system services are now being stopped.
syncing file systems... done
Program terminated
ChassisSerialNumber BEL07492JB

SPARC Enterprise T5120, No Keyboard
Copyright (c) 1998, 2017, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.33.6.h, 16256 MB memory available, Serial #78384094.
Ethernet address 0:14:4f:ac:b:de, Host ID: 84ac0bde.

{0} ok show-disks
a) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/disk
b) /pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/disk
c) /pci@0/pci@0/pci@2/scsi@0/disk
d) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
e) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@1/disk
f) /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk
g) /iscsi-hba/disk
q) NO SELECTION
Enter Selection, q to quit: d
/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
         for creating devalias mydev for /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk

{0} ok boot ^Y
{0} ok boot /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk
Boot device: /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/storage@2/disk  File and args:
'cpu:SUNW,UltraSPARC-T2:SUNW,sun4v-cpu:sun4v' is not supported by this release of Solaris.
Program terminated
ChassisSerialNumber BEL07492JB

SPARC Enterprise T5120, No Keyboard
Copyright (c) 1998, 2017, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.33.6.h, 16256 MB memory available, Serial #78384094.
Ethernet address 0:14:4f:ac:b:de, Host ID: 84ac0bde.

{0} ok

Note: The OpenSPARC T2 processor is not supported by Oracle Solaris 11.4, but Sun Microsystems Solaris 10, OpenSolaris, and Oracle Solaris 11.0 - Solaris 11.3 are all supported on the chassis.



Posted by at No comments:
Labels: , , , , , , , ,

Monday, May 17, 2021

Oracle Database Pre-Requisites Package

Oracle Database Pre-Requisites Package

Abstract:

The Oracle RDBMS Database has long been a tool on UNIX platforms. With the acquisition of Sun Microsystems, Oracle has drawn the Solaris operating system closer to it's orbit. One such change was to make a "pre-requisites" package in Solaris 11, to simplify installation of the RDBMS. There is a caution, if you already have standardized on user & group names for database usage.

Solaris Pre-Requisites

I was not going to write anything about this, but it appears that even other skilled engineers have run into problems with the automation of the Solaris Pre-Requisites package. I had seen a blog post from Alan, who graciously provided a solution when deploying hundreds of systems automatically.

sun9876/root# pkg install oracle-database-preinstall-19c

Alan struggled with ZFS Filesystems being created for new users by the pre-requisites script, before the user filesystems were mounted. This is not the only place where ZFS Filesystems being created for users are a problem, this author experienced several other conditions, one such condition kept a production system from coming up when the id's were scrambled by another such script.

This author filed a BUG to stop creating a ZFS File System for every user created. We have hundreds of users on some of our servers, so this ZFS feature was an inhibition to moving from Solaris 10 to 11. If you enable this feature in 11.4, you might be able to avoid the "avoid" that Alan had to use, if you don't mind the user & group id's being re-created.

Not all data centers deploy hundreds of Oracle DB's simultaneously. This author kept the pre-requisites script, but the default nature "blew away" our old user & group id's, adjusted ownership, and it was a nightmare. Procedures and scripts were built to undo everything that the pre-requisites package did with the id's & home dirctory, including returning the oracle user & dba groups back to their original ID's, correcting user & group ownership in /export/home, and then correcting the passwd, shadow, and group database. What a nightmare!

Conclusions

Honestly, it should be considered a BUG to rip out user & group names if those user & group names already exist, as a package overwrites them. After doing some DBA reading (this author is a systems guy first now a days) and do what the DBA's request of me, I found they missed a step which was to perform an "avoid" in order to avoid wiping out existing user & group id's for replacement, prior the preinstallation!

sun9876/root# pkg avoid oracle-database-os-configuration

There should NEVER be the need to perform an "avoid" from doing likely harm, but rather the package should be smart enough to realize id's exist and use them as defaults... and use an "avoid" clause, or something similar, so the "check" can be avoided and selectively allow the dangerous "blowing away & replacing" behavior.

Another piece of advise is to always perform a "dry-run" of an installation, before adding a package you are not aware of.

sun9876/root# pkg install -n oracle-database-preinstall-19c 

The dry run will show details that need to be understood, before applying an actual package later.


Posted by at No comments:
Labels: , , , , , , ,

Monday, February 1, 2021

Oracle 19c Installer: Root Equivalence Fails

 

Oracle 19c Installer: Root Equivalence Fails

Abstract:

The Oracle Installer is a common component for interactions with Oracle databases. It has a tendency of being a little buggy, very sensitive to the underlying operating system. When Solaris 11.4 upgraded, some of the underlying components are no longer compatible with the 19c installer, so workarounds must be implemented.

[Oracle RAC Architecture, courtesy Oracle Tutorial]

Oracle Architecture

When the installer is setting up the database in a RAC cluster, there is a procedure to set up root equivalence or test root equivalence. This is essentially password-less ssh between clustered nodes The process performs an "scp" of a file between the clustered hosts, and this can fail.

What can possibly go wrong?

A complete list of common failures and workarounds for the installer is located in an Oracle Note:
TOP Note: Solutions for Typical Grid Infrastructure/RAC Database runInstaller Issues (Doc ID 1056713.1)

The Oracle19c installer does not do "strict filename checking", which makes it incompatible with OpenSSH 8.x and newer since Versions 8.x and above enable “strict filename checking” by default. 

See Oracle Doc ID 2555697.1

Does it apply to my situation?

The scp binary must be wrapped with a script that calls the binary with a special compatibility flag.
(Note: fix will be undone as future “ssh” patches are applied and the workaround will need repeating if installer is needed in the future... which is why the procedure I provide below is important, so the wrapper script does not get purged during an upgrade.)

Check version of ssh to determine if system Oracle 19c is installing against is too new.

sun2202/oracle$ ssh -V
OpenSSH_8.1p1, OpenSSL 1.0.2u  20 Dec 2019

Note: The ssh software is too new for the Oracle 19c installer

Verify ssh is a binary and the workaround / fix has not already been implemented.

sun9999/oracle$ ls -alt /usr/bin/scp
-r-xr-xr-x   1 root     bin       135864 Jan  6 02:49 /usr/bin/scp

sun9999/oracle$ file /usr/bin/scp
/usr/bin/scp:       ELF 64-bit MSB dynamic lib SPARCV9 Version 1, UltraSPARC3 Extensions Required, position-independent executable, dynamically linked, not stripped, no debugging information available

Note: The scp command appears to be an original OS binary, meaning a wrapper can be applied.

What is the work around?

If the old Oracle 19c Installer is used, create shell wrapper to disable “strict filename checking.”

Create the wrapper and check permissions & ownership.

sun9999/root# cat /usr/bin/scp.Doc.ID.2555697.1

#
# bug in oracle installer, for compatibility with OpenSSH 8.x
# INS-06006 GI RunInstaller Fails If OpenSSH Is Upgraded to 8.x
# (Doc ID 2555697.1)
#
/usr/bin/scp.orig -T $* 

sun9999/root# ls -al /usr/bin/scp.Doc.ID.2555697.1
-r-xr-xr-x 1 root  bin   209 Jun 15  2020 /usr/bin/scp.Doc.ID.2555697.1

Note: the shell wrapper above was created, disables checking, and has corret permissions & ownership.

Show Binaries & Shell Wrapper

sun9999/root# ls -alt /usr/bin/scp*
-r-xr-xr-x 1 root bin 135864 Jan  6 02:49 /usr/bin/scp
-r-xr-xr-x 1 root bin    209 Jun 15  2020 /usr/bin/scp.Doc.ID.2555697.1

Copy binary to “.orig” for Wrapper, Move binary to backup [by OS patch], and Copy Wrapper in place.

sun9999/root# uname -a
SunOS sun2202 5.11 11.4.28.82.3 sun4v sparc sun4v

sun9999/oracle$ Backup=/usr/bin/scp.11.4.28

sun9999/oracle$ echo $Backup
/usr/bin/scp.11.4.28

sun9999/root# cp -p  /usr/bin/scp /usr/bin/scp.orig
sun9999/root# mv     /usr/bin/scp $Backup
sun9999/root# scp -p /usr/bin/scp.Doc.ID.2555697.1 /usr/bin/scp

sun9999/root# chown root:bin     /usr/bin/scp
sun9999/root# chmod 555          /usr/bin/scp

Show Binaries & Shell Wrapper

sun9999/root# ls -alt /usr/bin/scp*
-r-xr-xr-x 1 root bin    209 Jan 19 16:23 /usr/bin/scp
-r-xr-xr-x 1 root bin 135864 Jan  6 02:49 /usr/bin/scp.11.4.28
-r-xr-xr-x 1 root bin 135864 Jan  6 02:49 /usr/bin/scp.orig
-r-xr-xr-x 1 root bin    209 Jun 15  2020 /usr/bin/scp.Doc.ID.2555697.1

Verify scp script is functional

sun9999/oracle$ type scp
scp is hashed (/usr/bin/scp)

sun9999/oracle$ scp
usage: scp [-346BCpqrTv] [-c cipher] [-F ssh_config] [-i identity_file]
            [-J destination] [-l limit] [-o ssh_option] [-P port]
            [-S program] source ... target

Instruct the DBA’s to resume use of the Oracle 19c installer

Caveats:

When an upgrade happens, it will be important to identify if the "scp" command is no longer a script and has been reverted to the binary.

If this had occurred, follow the same steps above:

1. creating a new ".orig"
2. create a new backup of the binary tagging it by what OS release & SRU
3. copy the wrapper back into place, with proper ownership & permissions

With the old wrapper & backups in place, you should be able to figure out what needs to be done without trying to find these instructions again.

Posted by at 1 comment:
Labels: , , , , , , ,
Subscribe to: Posts (Atom)