Patching fails with "Linux x86-64 that is not one of the supported platform"

The 19.12 patch cycle was out last week for both Database and Grid Infrastructure. Since then, I've seen some customers complaining about an issue in the tomcat patch when trying to patch their environment with the latest opatch version:

The oracle home /u01/app/grid has a platform  Linux x86-64 that is not one of the supported platform for the given patch: [ Generic Platform]
OPATCHAUTO-72009: Invalid ARU id for the platform.
OPATCHAUTO-72009: ARU id value not recognized as a valid id for the platform.
OPATCHAUTO-72009: Check system configuration.
OPatchAuto failed.

In this post, I will try to simulate the issue and show how to fix it.

So, first I downloaded both GI 19.12 and the latest opatch:

• Patch 32895426 - GI Release Update 19.12.0.0.210720
• https://updates.oracle.com/download/6880880.html

In GI 19.12 README, we have under "1.2.1.1 OPatch Utility Information":

So I'm using the required version:

[root@node1 GI_19_12]# opatchauto version
Oracle OPatchAuto Version 12.2.1.25.0
Copyright (c) 2016, Oracle Corporation.  All rights reserved.

1. OPatchAuto version 12.2.1.25.0
2. OpatchautoDB version 12.2.0.1.25

Let me start patching right away.

PS: Note that I always recommend OOP (Out-of-Place) patching, as you reduce downtime and also the risk of issues. However, here I will use in-place to show what to do when you get stuck in the middle with one of your nodes down.

[root@node1 GI_19_12]# opatchauto apply ./32895426/ -oh /u01/app/grid

OPatchauto session is initiated at Thu Aug 12 09:55:51 2021

System initialization log file is /u01/app/grid/cfgtoollogs/opatchautodb/systemconfig2021-08-12_09-56-12AM.log.

Session log file is /u01/app/grid/cfgtoollogs/opatchauto/opatchauto2021-08-12_09-56-33AM.log
The id for this session is 1L1V

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid
Patch applicability verified successfully on home /u01/app/grid


Executing patch validation checks on home /u01/app/grid
Patch validation checks successfully completed on home /u01/app/grid


Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/grid
Prepatch operation log file location: /gihome/crsdata/node1/crsconfig/crs_prepatch_apply_inplace_node1_2021-08-12_09-57-47AM.log
CRS service brought down successfully on home /u01/app/grid


Start applying binary patch on home /u01/app/grid
Binary patch applied successfully on home /u01/app/grid


Performing postpatch operations on CRS - starting CRS service on home /u01/app/grid
Postpatch operation log file location: /gihome/crsdata/node1/crsconfig/crs_postpatch_apply_inplace_node1_2021-08-12_10-15-48AM.log
CRS service started successfully on home /u01/app/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:node1
CRS Home:/u01/app/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/GI_19_12/32895426/32585572
Log: /u01/app/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-08-12_10-02-26AM_1.log

Patch: /home/oracle/GI_19_12/32895426/32904851
Log: /u01/app/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-08-12_10-02-26AM_1.log

Patch: /home/oracle/GI_19_12/32895426/32915586
Log: /u01/app/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-08-12_10-02-26AM_1.log

Patch: /home/oracle/GI_19_12/32895426/32916816
Log: /u01/app/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-08-12_10-02-26AM_1.log

Patch: /home/oracle/GI_19_12/32895426/32918050
Log: /u01/app/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-08-12_10-02-26AM_1.log



OPatchauto session completed at Thu Aug 12 10:21:25 2021
Time taken to complete the session 25 minutes, 34 seconds

So as you can see I haven't had any issues. After researching a bit more, I found out this was an issue with "opatchauto resume", not with "opatchauto apply". And more specific to the opatch version ".25", the one required by this GI.

So let's forcibly introduce an error and try again the "opatchauto resume" in a new environment to simulate this issue. Here what I did was removing the execution flag from the java binary that will be shipped to the GRID_HOME. So this way java will fail to execute after the node is patched:

[root@node1 GI_19_12]# ls -la 32895426/32904851/files/jdk/bin/java
-rwxr-xr-x 1 oracle oinstall 8712 Jul 20 16:42 32895426/32904851/files/jdk/bin/java
[root@node1 GI_19_12]# chmod -x 32895426/32904851/files/jdk/bin/java
[root@node1 GI_19_12]# ls -la 32895426/32904851/files/jdk/bin/java
-rw-r--r-- 1 oracle oinstall 8712 Jul 20 16:42 32895426/32904851/files/jdk/bin/java

Now if I try opatchauto apply:

[root@node1 GI_19_12]# opatchauto apply ./32895426/ -oh /u01/app/grid

OPatchauto session is initiated at Thu Aug 12 13:20:28 2021

System initialization log file is /u01/app/grid/cfgtoollogs/opatchautodb/systemconfig2021-08-12_01-20-39PM.log.

Session log file is /u01/app/grid/cfgtoollogs/opatchauto/opatchauto2021-08-12_01-21-15PM.log
The id for this session is 7SDU

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid
Patch applicability verified successfully on home /u01/app/grid


Executing patch validation checks on home /u01/app/grid
Patch validation checks successfully completed on home /u01/app/grid


Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/grid
Prepatch operation log file location: /gihome/crsdata/node1/crsconfig/crs_prepatch_apply_inplace_node1_2021-08-12_01-22-38PM.log
CRS service brought down successfully on home /u01/app/grid


Start applying binary patch on home /u01/app/grid
Failed while applying binary patches on home /u01/app/grid

Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : node1->/u01/app/grid Type[crs]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/grid, host: node1.
Command failed:  /u01/app/grid/OPatch/opatchauto  apply ./32895426/ -oh /u01/app/grid -target_type cluster -binary -invPtrLoc /u01/app/grid/oraInst.loc -jre /u01/app/grid/OPatch/jre -persistresult /u01/app/grid/opatchautocfg/db/sessioninfo/sessionresult_node1_crs_14.ser -analyzedresult /u01/app/grid/opatchautocfg/db/sessioninfo/sessionresult_analyze_node1_crs_14.ser
Command failure output:
==Following patches FAILED in apply:

Patch: /home/oracle/GI_19_12/32895426/32904851
Log: /u01/app/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-08-12_13-25-19PM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: ApplySession failed in system modification phase... 'ApplySession::apply failed: /u01/app/grid/bin/patchgen: line 138: /u01/app/grid/jdk/bin/java: Permission denied
'

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Thu Aug 12 13:30:57 2021
Time taken to complete the session 10 minutes, 30 seconds

 opatchauto failed with error code 42

It failed with "ApplySession::apply failed: /u01/app/grid/bin/patchgen: line 138: /u01/app/grid/jdk/bin/java: Permission denied" as expected.

Let's try to fix back the java permission and retry:

[root@node1 GI_19_12]# ls -la 32895426/32904851/files/jdk/bin/java
-rw-r--r-- 1 oracle oinstall 8712 Jul 20 16:42 32895426/32904851/files/jdk/bin/java
[root@node1 GI_19_12]# chmod +x 32895426/32904851/files/jdk/bin/java
[root@node1 GI_19_12]# ls -la 32895426/32904851/files/jdk/bin/java
-rwxr-xr-x 1 oracle oinstall 8712 Jul 20 16:42 32895426/32904851/files/jdk/bin/java
[root@node1 GI_19_12]#

I will also fix the java file that was already shipped to the GRID_HOME:

[root@node1 GI_19_12]# ls -la $ORACLE_HOME/jdk/bin/java
-rw-r--r-- 1 oracle oinstall 8712 Jul 20 16:42 /u01/app/grid/jdk/bin/java
[root@node1 GI_19_12]# chmod +x $ORACLE_HOME/jdk/bin/java
[root@node1 GI_19_12]# ls -la $ORACLE_HOME/jdk/bin/java
-rwxr-xr-x 1 oracle oinstall 8712 Jul 20 16:42 /u01/app/grid/jdk/bin/java
[root@node1 GI_19_12]#

Retrying:

[root@node1 GI_19_12]# opatchauto resume

OPatchauto session is initiated at Thu Aug 12 13:32:34 2021
Session log file is /u01/app/grid/cfgtoollogs/opatchauto/opatchauto2021-08-12_01-32-38PM.log
Resuming existing session with id 7SDU
The oracle home /u01/app/grid has a platform  Linux x86-64 that is not one of the supported platform for the given patch: [ Generic Platform]
OPATCHAUTO-72009: Invalid ARU id for the platform.
OPATCHAUTO-72009: ARU id value not recognized as a valid id for the platform.
OPATCHAUTO-72009: Check system configuration.
OPatchAuto failed.

OPatchauto session completed at Thu Aug 12 13:33:08 2021
Time taken to complete the session 0 minute, 34 seconds

 opatchauto failed with error code 42

Now as you can see we are stuck in this failure where we can't resume. There is a MOS Note for it:

Basically, this is an issue introduced on opatchauto ".25" and will be fixed on the ".27", that happens when trying to resume and when the Patch has a generic platform.

The MOS note suggests 2 ways for solving it:

1. Restore from backup older OPatch version .24 or .23 and invoke OPatchauto resume.
2. If there is no backup older version of OPatch then we can apply the patch manually via OPatch apply against <GI_HOME> and when done we must run the below to run the post steps in order to lock <GI_HOME> and bring the CRS stack.

As I don't have a previous ".24" patch to try, the best option here would be to try the second method: applying this specific patch that failed (in my case it was the 32918050) manually.

However, applying a patch manually in GI is not that simple. You may need to stop your CRS / unlock grid home / do it in multiple nodes / etc. There is even a MOS note for it. In step 5 of note below, you have all the details:

What I rather prefer to do is just restart the GI back and retry "opatchauto apply" instead of using the bugged "opatchauto resume":

[root@node1 GI_19_12]# $ORACLE_HOME/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@node1 GI_19_12]# opatchauto apply ./32895426/ -oh /u01/app/grid

And it succeeds. That is it. Here are my 50 cents with a third option to solve the stuck GI patch apply. Just keep in mind that If DB is involved, then also start the database using "srvctl start database".

+4