Oracle Enterprise Manager – WebTier / OHS Fails to Start
Introduction
A common issue in Oracle Enterprise Manager Cloud Control (OEM 13c) is the sudden failure of the WebTier (Oracle HTTP Server – OHS) component. When this happens, the OMS fails to start completely and shows the error:
WebTier Could Not Be Started.
This problem is frequently caused by an expired self-signed certificate in the OHS default keystore. The certificate is valid for only 5 years by default, and once it expires, OHS cannot initialize the SSL environment.
This article explains the symptoms, root cause, how to confirm the expiry, and the complete steps to resolve the issue.
Symptoms
When you try to start the OMS:
Bash
emctl start oms
You receive the following error:
Starting Oracle Management Server…
WebTier Could Not Be Started.
Error Occurred: WebTier Could Not Be Started.
Relevant log entries:
emctl.log
Failed to start OHS component named ohs1
Exception while starting OHS: … Error Starting server ohs1
WebTier Could Not Be Started.
ohs1.log
OHS:2057 Init: (localhost:443) Unable to initialize SSL environment,
nzos call nzosSetCredential returned 28791
OHS:2171 NZ Library Error: Unknown error
Error 28791 maps to a certificate verification failure.
Root Cause
From WebLogic 12.2.1.3.0 onwards, the OHS Admin port (default 10006 / 443) is SSL-enabled by default and uses a self-signed certificate stored in:
$GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/keystores/default/cwallet.sso
This certificate is created with a 5-year validity. Once it expires, OHS fails to start.
Example of an expired certificate:
Validity
Not Before: Jun 9 16:42:30 2021 GMT
Not After : Jun 8 16:42:30 2026 GMT ← Expired
How to Confirm the Certificate Has Expired
-
Go to the keystore location:
cd $GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/keystores/default
-
Display the wallet:
$OMS_HOME/oracle_common/bin/orapki wallet display -wallet .
-
Export the user certificate:
$OMS_HOME/oracle_common/bin/orapki wallet export \
-wallet . \
-dn “CN=localhost,OU=GCDomain ohs1,O=FOR TESTING ONLY” \
-cert /tmp/server.cer
-
Check the validity using OpenSSL:
openssl x509 -in /tmp/server.cer -noout -text | grep -A2 Validity
If the Not After date is in the past, the certificate has expired.
Solution – Regenerate the Self-Signed Certificate
Follow these steps carefully.
Step 1: Backup the existing wallet
cd $GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/keystores/default
mv cwallet.sso cwallet.sso.expired
Step 2: Create a new blank wallet
$OMS_HOME/oracle_common/bin/orapki wallet create \
-wallet $GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/keystores/default \
-auto_login_only
Important: Always use orapki from $OMS_HOME/oracle_common/bin, not from $OMS_HOME/bin.
Step 3: Add a new self-signed certificate (valid for 10 years)
$OMS_HOME/oracle_common/bin/orapki wallet add \
-wallet $GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/keystores/default \
-dn ‘CN=localhost,OU=GCDomain ohs1,O=FOR TESTING ONLY’ \
-keysize 2048 \
-self_signed \
-validity 3650 \
-sign_alg sha256 \
-auto_login_only
Step 4: Verify the new certificate
$OMS_HOME/oracle_common/bin/orapki wallet display \
-wallet $GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/keystores/default
Step 5: Copy the new wallet to the Instance location
cp $GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/keystores/default/cwallet.sso \
$GC_INST/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/instances/ohs1/keystores/default/
Critical Note: Always perform the wallet creation on the Stage location and then copy it to the Instance location.
Step 6: Restart the OMS
$OMS_HOME/bin/emctl stop oms -all -force
$OMS_HOME/bin/emctl start oms
Multi-OMS Environment
If you have additional OMS nodes, you must also update the wallet on those nodes.
-
On the Primary OMS, copy the newly created cwallet.sso to a temporary location.
-
Transfer it to each additional OMS host.
-
On each additional OMS:
-
-
Backup the existing cwallet.sso under the corresponding ohsN instance.
-
Copy the new cwallet.sso into the instance keystore location.
-
Restart the additional OMS.
-
Important Notes
-
The certificate generated is still a self-signed certificate (for testing/internal use).
-
Validity is set to 3650 days (10 years) in the above command.
-
Always use the Stage location for wallet creation.
-
Make sure the orapki command is executed from $OMS_HOME/oracle_common/bin.
Conclusion
The “WebTier Could Not Be Started” error in Oracle Enterprise Manager is most often caused by an expired OHS default certificate. Because the default self-signed certificate has only a 5-year lifespan, this issue will eventually appear in every OEM environment.
By regenerating the certificate using orapki and properly updating both the Stage and Instance locations, you can quickly restore the OMS WebTier and bring the environment back online.
