Showing posts with label Solaris 11. Show all posts
Showing posts with label Solaris 11. Show all posts

Friday, February 17, 2023

NAWK: How can I get uptime & boot time in Solaris?

 

[Solaris Logo, courtesy former Sun Microsystems, Oracle]

Abstract:

This is part of a small article series on lesser known uses of UNIX nawk. Why nawk? This POSIX language has not changed in decades, gets very few code changes, and is mostly bug free in any reasonable operating system. Scripts written in POSIX nawk will basically run, forever. Master this simple language, and there is virtually nothing that can not be done.

What is AWK?

Awk is a scriptable text language, originally written by the authors of UNIX and C: Alfred Aho, Peter Weinberger, and Brian Kernighan. It's concepts are managing Page Header (the BEGIN clause), Page Body (multiple Pattern -> Action clauses), and Page Footer (the END clause.) 

Awk is suitable for building & manipulating any textual or markup text, which is page oriented (i.e. SQL output, Text, Adobe PostScript, Adobe FrameMaker portable maker format, HTML, XML, Microsoft RTF, Microsoft CSV, etc.)

NAWK: How can I get uptime & boot time in Solaris?

The CTime or Epoch Time can be easily retrieved from nawk using the "srand()" function, if there is no seed passed into the function. This number represents the seconds since 00:00:00 on January 1, 1970.

The boot time is the CTime/Epoch Time [in seconds], at the time of the last OS boot, and can be retrieved from "kstat". Simple subtraction and division can provide reasonably accurate conversions.

The following examples show: CTime, Boot Time, Seconds Difference, Minutes Difference, Hours Difference, Days Difference.

Example 1 - Solaris 10:

Solaris10/root# kstat -n system_misc | nawk '
BEGIN { CTime=srand() };
/boot_time/ { BTime=$2; Diff=CTime-BTime };
END { printf "CTime:\t%s\nBTime:\t%s\nScDif:\t%s\nMnDif:\t%s\nHrDif:\t%s\nDyDif:\t%s\n",
      CTime,BTime,Diff,Diff/60,Diff/60/60,Diff/60/60/24 }'

CTime:  1676683570
BTime:  1676106161
ScDif:  577409
MnDif:  9623.48
HrDif:  160.391
DyDif:  6.68297

Example 2 - Solaris 11:

Solaris 11/root# kstat -n system_misc | nawk '
BEGIN { CTime=srand() };
/boot_time/ { BTime=$2; Diff=CTime-BTime };
END { printf "CTime:\t%s\nBTime:\t%s\nScDif:\t%s\nMnDif:\t%s\nHrDif:\t%s\nDyDif:\t%s\n",
      CTime,BTime,Diff,Diff/60,Diff/60/60,Diff/60/60/24 }'

CTime:  1676683533
BTime:  1662691183
SecDif: 13992350
MinDif: 233206
HrDif:  3886.76
DayDif: 161.948

Does the code look similar? It should, Solaris 10 and Solaris 11 are POSIX compliant.

What about GAWK?

GNU's Awk is not 100% compatible with POSIX Nawk, which is a huge disappointment. I have seen code, which worked for decades, never able to run unchanged under GNU's awk.

 

Monday, October 4, 2021

Differences Between Solaris 10 and Solaris 11.4

 

Differences Between Solaris 10 and Solaris 11.4

Abstract:

Sun Microsystems used to migrate between operating systems fairly regularly. A new trend had come to the software development community referred to as Continuous Delivery. Oracle purchased Sun Microsystems. Solaris 10 acquired many new features, the life expectancy was extended significantly, and Solaris 11 was released mid-way through the significantly lengthened support cycle. Instead of releasing Solaris 12, Oracle made the executive decision to roll all features of Solaris 12 into Solaris 11.4.

Solaris 11.4 aka Solaris 12

What are some of the differences between Solaris 10 and Solaris 11.4?

Oracle published a document summarizing the differences, with links to major documents.

Key Differences between Oracle Solaris 10 and Oracle Solaris 11

Upgrading from Oracle Solaris 10 to Oracle Solaris 11 requires a fresh installation of Oracle Solaris 11.

Tools to help you make the transition include the following:

  • Oracle Solaris 10 branded zones. Migrate Oracle Solaris 10 instances to Oracle Solaris 10 zones on Oracle Solaris 11 systems.

  • ZFS shadow migration. Migrate UFS data from an existing file system, either local or NFS, to a new local ZFS file system. Do not mix UFS directories and ZFS file systems in the same file system hierarchy.

    You can also remotely mount UFS file systems from an Oracle Solaris 10 system onto an Oracle Solaris 11 system, or use the ufsrestore command on an Oracle Solaris 10 system to restore UFS data (ufsdump) into an Oracle Solaris 11 ZFS file system.

  • ZFS pool import. Export and disconnect storage devices that contain ZFS storage pools on your Oracle Solaris 10 systems and then import them into your Oracle Solaris 11 systems.

  • NFS file sharing. Share files from an Oracle Solaris 10 system to an Oracle Solaris 11 system. Do not mix NFS legacy shared ZFS file systems and ZFS NFS shared file systems. Use only ZFS NFS shared file systems.

For the main Oracle Solaris documentation, see Oracle Solaris Documentation. For additional documentation and examples, select a technology on the Oracle Solaris 11 Technology Spotlights page.

Applications that run on Oracle Solaris 10 should also run on Oracle Solaris 11 if they use only public Oracle Solaris interfaces. Oracle Solaris Preflight Applications Checker 11.3 can determine the Oracle Solaris 11 readiness of an application by analyzing the working application on Oracle Solaris 10. A successful check with this tool strongly indicates that you can run the application without modification on Oracle Solaris 11.

Versions of FOSS and other software are updated. In some cases, a system can have more than one version of a command or tool simultaneously installed. If your application depends on a particular version, use the full path to the executable rather than depend on a link.

See End of Feature Notices for Oracle Solaris 11 for lists of commands and tools that are no longer available in Oracle Solaris 11. In most cases, Oracle Solaris 11 provides alternative commands and tools. The list also includes hardware that does not support newer Oracle Solaris 11 versions.

A graphical desktop is not included by default with some system installations. If you want a graphical desktop, install the group/system/solaris-desktop IPS package.

Installation and Upgrade Changes

The following are key changes from Oracle Solaris 10 to Oracle Solaris 11:

  • Installation and upgrade:

    • Instead of JumpStart, use Automated Installer.

    • Instead of Live Upgrade, use the text installer or Image Packaging System (IPS) pkg commands.

    • Software packages are delivered in package repositories, similar to Linux package repositories.

  • Archive and recovery: Instead of Flash Archives, use Unified Archives.

  • System services: More system configuration is done by setting Service Management Facility (SMF) service property values and not by directly editing configuration files. Look for comments in the configuration files and see the documentation for that configuration.

  • root user: By default, root is a role, not a user. Instead of doing privileged tasks as root, create and assign roles targeted to each set of related tasks.

  • Shell: The default shell for the root user is ksh. The default shell for other users is bash. Default user PATH also has changed.

Changes in How to Configure Oracle Solaris Features

More configuration is provided by partial configuration files in the /etc/system.d directory, where customer-specific system configuration files should also be stored. Routinely editing /etc/system should be avoided. In some cases, the partial configuration file is created by an SMF service using service property values that you provide.

For network configuration, Oracle Solaris 11 assigns generic names to each datalink on a system by using the net0, net1, netN naming convention. Configuration is also managed through SMF service property values rather than by directly editing configuration files. In addition, new commands for setting up datalinks and IP interfaces have been introduced to replace the commonly used commands in Oracle Solaris 10, such as ifconfig.

Networking in Oracle Solaris 11 has advanced to provide better network performance, efficient network resource management, higher network availability, and new technologies such as in the area of network virtualization. See the documentation in Administering Oracle Solaris Networks and Administering Network Services in Oracle Solaris.

Changes in User Environment

  • Default login and other shell changes - In Oracle Solaris 11, /bin/sh is the Korn shell (ksh93), and the default interactive shell is the Bourne-again (bash) shell. When used as a login shell, bash retrieves configuration information from the first instance of .bash_profile, .bash_login, or .profile file.

    • The legacy Bourne shell is available as /usr/sunos/bin/sh.

    • The legacy ksh88 is available as /usr/sunos/bin/ksh from the shell/ksh88 package.

    • Korn shell compatibility information is available in /usr/share/doc/ksh/COMPATIBILITY.

  • Default user path and PATH environment variable – The default user path is /usr/bin. The default path for the root role is /usr/bin:/usr/sbin. The default PATH environment variable for bash is /usr/bin:/usr/sbin

For more details about user environment in Oracle Solaris 11.4, see About the User Work Environment in Managing User Accounts and User Environments in Oracle Solaris 11.3.

Changes in Security

Security in Oracle Solaris 11 supports industry standards more closely. For an overview of security in Oracle Solaris 11, see Security: An Oracle Solaris Differentiator.

Other enhancements increase hardening, add compliance functionality, and enable remote administration of security:

Tuesday, January 26, 2021

Solaris 11.4 - Oracle ZFS Buffer Cache and App Memory Hints

 

Solaris 11.4 - Oracle ZFS Buffer Cache and App Memory Hints

Abstract:

Solaris has long used the UFS File System as the default for root disks. This filesystem placed I/O buffers in the "free space", causing misleading performance problems for Disk I/O when there was seemingly plenty of memory left. This was remedied by ZFS, where disk I/O buffers are counted as used buffer, truly showing only "wasted" memory as free memory. This causes a problem for some applications (i.e. Oracle RDBMS Installers), which attempt to determine how much free space is available before performing an application installation. There is a "work around" for these broken applications.

Background:

An excellent document to understand the use of memory is available from Oracle:
Memory Management Between ZFS and Applications in Oracle Solaris 11.x (Doc ID 1663862.1)

It is not the objective of this blog to re-explain this document, merely show what has not been included by Oracle, for manually making adjustments to the memory usage.

Determining Memory Usage:

One can determine the memory available for any version of Solaris with:

sun9999/root# prtconf | nawk '/Memory/'
Memory size: 32768 Megabytes

And determine the usage by different aspects of Solaris with:

sun9999/root# echo "::memstat" | mdb -k

Usage Type/Subtype                      Pages    Bytes  %Tot  %Tot/%Subt
---------------------------- ---------------- -------- ----- -----------

Kernel                                 585907    4.47g 13.9%

 
Regular Kernel                       432874     3.3g       10.3%/73.8%

 
Defdump prealloc                     153033    1.17g        3.6%/26.1%

ZFS                                   3063918    23.4g 73.0%

User/Anon                              269262    2.05g  6.4%

Exec and libs                            4369    34.1m  0.1%

Page Cache                              20525     160m  0.4%

Free (cachelist)                        14076     110m  0.3%

Free                                   235161    1.79g  5.6%

 
Regular Free                         232151    1.77g        5.5%/98.7%

 
Physical pool reserved mem             3010    23.5m        0.0%/ 1.2%

Total                                 4194304      32g  100%

Note: 5.6% is wasted memory, out of the 32Gig total, but some 23.4G /73% can be used, if requested.

The application must request the memory, for it to be available, if it can not interrogate now much it can request from the kernel.

What to Do if an App Fails to Start

Option 1: Reboot

If an application fails to start, like an Oracle RDBMS Installer, because it is not smart enough to determine most of the memory is free, you can temporarily flush the ZFS Adaptive Read Cache (ARC) by rebooting the boot.

The system will flush the cache and there will be plenty of memory available for the installer.

Option 2: Dynamically Tune

Oracle Solaris 11.4 created a new tunable called  "user_reserve_hint_pct", which requests the OS to dynamically reserve a percentage of the memory for the application by flushing the ARC.

There is a danger: do not reserve more memory for the user processes than the Kernel & OS require!

Step 1: Determine default value (should be 0)

sun9999/root# echo "user_reserve_hint_pct/D" | mdb -k
user_reserve_hint_pct:

user_reserve_hint_pct:          0

 Step 2: Select value from determined memory usage (ZFS 73 - Free 5.6 = ~67%, round down 65%)

sun9999/root# echo "user_reserve_hint_pct/W0t65" | mdb -k
user_reserve_hint_pct:

user_reserve_hint_pct:          65


sun9999/root# echo "::memstat" | mdb -k

Usage Type/Subtype                      Pages    Bytes  %Tot  %Tot/%Subt
---------------------------- ---------------- -------- ----- -----------

Kernel                                 599572    4.57g 14.2%

 
Regular Kernel                       446539    3.41g       10.6%/74.4%

 
Defdump prealloc                     153033    1.17g        3.6%/25.5%

ZFS                                    547332    4.18g 13.0%

User/Anon                              268734    2.05g  6.4%

Exec and libs                            4359    34.1m  0.1%

Page Cache                              20190     158m  0.4%

Free (cachelist)                        13903     109m  0.3%

Free                                  2739134    20.9g 65.3%

 
Regular Free                        2736118    20.9g       65.2%/99.8%

 
Physical pool reserved mem             3016    23.6m        0.0%/ 0.1%

Total                                 4194304      32g  100%

Step 3: Revert Value to Default since much of the the ARC cache has been flushed.

sun9999/root# echo "user_reserve_hint_pct/W0t0" | mdb -kw
user_reserve_hint_pct:          0x41            =       0x0

sun9999/root# echo "user_reserve_hint_pct/D" | mdb -k
user_reserve_hint_pct:

user_reserve_hint_pct:          0

At this point, the application should be able to be dynamically tuned.

Conclusion:

Oracle Solaris provides an incredibly tunable environment for running applications, including legacy applications, which are not quite smart enough to understand the benefits of the underlying OS.