GI Release Update 19.9.0.0.201020″(31750108) patch on GI Home of one of servers.
I used opatchauto utility to apply this patch. Following command was run for the same:
$<GI_HOME>/OPatch/opatchauto apply <PATCH DIRECTORY> -oh <GI_HOME>
After waiting for the sometime, I found my command was stuck without any progress. It was not moving further after printing below status information:
OPatchauto session is initiated at Sat Dec 12 12:23:19 2020 System initialization log file is /u01/app/19.6.0.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-23-24-52PM.log. Session log file is /u01/app/19.6.0.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-12_03-24-19PM.log The id for this session is JKL8 Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.6.0.0/grid Patch applicability verified successfully on home /u01/app/19.6.0.0/grid Bringing down CRS service on home /u01/app/19.6.0.0/grid Prepatch operation log file location: /u01/app/oracle/crsdata/test-server-1/crsconfig/hapatch_2020-12-12_03-26-03PM.log CRS service brought down successfully on home /u01/app/19.6.0.0/grid Start applying binary patch on home /u01/app/19.6.0.0/grid Successfully executed command: /usr/sbin/slibclean
I checked lot many relevant logfiles associated with this opatchauto session, I found none of were updated with proper information what the session is currently doing.
So after traversing in between multiple directories & files I found following logfile which has information on what the session is waiting for:
/u01/app/19.6.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-12_23-24-52PM_1.log
Logfile had repeated warnings as given below:
Do you want to proceed? [y|n] [Dec 12, 2020 3:28:55 PM] [INFO] Y (auto-answered by -silent) [Dec 12, 2020 3:28:55 PM] [INFO] User Responded with: Y [Dec 12, 2020 3:28:55 PM] [INFO] OPatch will sleep for few seconds, before re-trying to get the lock… [Dec 12, 2020 3:30:55 PM] [INFO] Unable to lock Central Inventory. OPatch will attempt to re-lock. Do you want to proceed? [y|n] [Dec 12, 2020 3:30:55 PM] [INFO] Y (auto-answered by -silent) [Dec 12, 2020 3:30:55 PM] [INFO] User Responded with: Y [Dec 12, 2020 3:30:55 PM] [INFO] OPatch will sleep for few seconds, before re-trying to get the lock… [Dec 12, 2020 3:32:55 PM] [INFO] Unable to lock Central Inventory. OPatch will attempt to re-lock.
I found the culprit. Patching session was unable to lock central inventory. After checking I found locks directory had incorrect ownership. After changing its ownership back to Oracle user, patching session moved further.
Following commands were used to fix the issue:
As a root user: cd <oraInventory location> chown -R oracle:oinstall locks