Installing 18c (18.3) RAC on RHEL 7 with Role Separation - Database

Following the 18c clusterware installation the next step is to install database software and create the database. The database software installation is not much different from that of 12.2 database software installation. Even though the oracle installation guides states gcc and gcc-c++ are not required for RHEL, the cluvfy still searches for gcc-c++.
1. Create the Oracle home for the database software and unzip the installer into the directory.

cd $ORACLE_BASE
mkdir -p product/18.x.0/dbhome_1
unzip LINUX.X64_180000_db_home.zip -d /opt/app/oracle/product/18.x.0/dbhome_1/

2. Run cluvfy to check the pre dbinst state. The failures related to shm could be ignored

cluvfy stage -pre dbinst -allnodes -r 18

Verifying Physical Memory ...PASSED
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...PASSED
Verifying Free Space: rhel72:/tmp ...PASSED
Verifying Free Space: rhel71:/tmp ...PASSED
Verifying User Existence: oracle ...
  Verifying Users With Same UID: 501 ...PASSED
Verifying User Existence: oracle ...PASSED
Verifying Group Existence: dba ...PASSED
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Group Membership: dba ...PASSED
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying Package: binutils-2.23.52.0.1 ...PASSED
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: libgcc-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...PASSED
Verifying Package: sysstat-10.1.5 ...PASSED
Verifying Package: gcc-c++-4.8.2 ...PASSED
Verifying Package: ksh ...PASSED
Verifying Package: make-3.82 ...PASSED
Verifying Package: glibc-2.17 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.17 (x86_64) ...PASSED
Verifying Package: libaio-0.3.109 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.109 (x86_64) ...PASSED
Verifying Package: smartmontools-6.2-4 ...PASSED
Verifying Package: net-tools-2.0-0.17 ...PASSED
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...PASSED
Verifying Package: libxcb-1.11 (x86_64) ...PASSED
Verifying Package: libX11-1.6.3 (x86_64) ...PASSED
Verifying Package: libXau-1.0.8 (x86_64) ...PASSED
Verifying Package: libXi-1.7.4 (x86_64) ...PASSED
Verifying Package: libXtst-1.2.2 (x86_64) ...PASSED
Verifying Users With Same UID: 0 ...PASSED
Verifying Current Group ID ...PASSED
Verifying Root user consistency ...PASSED
Verifying Host name ...PASSED
Verifying Node Connectivity ...
  Verifying Hosts File ...PASSED
  Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
  Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
  Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast or broadcast check ...PASSED
Verifying User Mask ...PASSED
Verifying CRS Integrity ...
  Verifying Clusterware Version Consistency ...PASSED
Verifying CRS Integrity ...PASSED
Verifying Cluster Manager Integrity ...PASSED
Verifying Node Application Existence ...PASSED
Verifying Clock Synchronization ...PASSED
Verifying resolv.conf Integrity ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Single Client Access Name (SCAN) ...
  Verifying DNS/NIS name service 'rac-scan.domain.net' ...
    Verifying Name Service Switch Configuration File Integrity ...PASSED
  Verifying DNS/NIS name service 'rac-scan.domain.net' ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verifying VIP Subnet configuration check ...PASSED
Verifying Database Clusterware Version Compatibility ...PASSED
Verifying ASM storage privileges for the user: oracle ...
  Verifying Group Membership: asmdba ...PASSED
Verifying ASM storage privileges for the user: oracle ...PASSED
Verifying Daemon "proxyt" not configured and running ...PASSED
Verifying ACFS device special file ...PASSED
Verifying /dev/shm mounted as temporary file system ...FAILED (PRVE-0421)
Verifying Maximum locked memory check ...PASSED

Pre-check for database installation was unsuccessful on all the nodes.


Failures were encountered during execution of CVU verification request "stage -pre dbinst".

Verifying /dev/shm mounted as temporary file system ...FAILED
rhel72: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

rhel71: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

3. Execute runInstaller from the Oracle home and select software only install option. Other steps are similar to 12.2. However, one noticeable change is that in 18c the space available for Oracle base is also checked as part of pre-reqs. Any warning raised for test system could be ignored. 4. Once the database software is installed proceed to creating the database. As this is a role separate setup, certain folders need write permission for the oinstall group so oracle user could write into them.

ls -l
total 0
drwxr-x---.  5 grid oinstall  45 Mar 20 11:50 admin
drwxr-x---.  2 grid oinstall   6 Mar 20 11:51 audit

[grid@rhel71 oracle]$ chmod 770 admin audit

[grid@rhel71 oracle]$ ls -l
total 0
drwxrwx---.  5 grid oinstall  45 Mar 20 11:50 admin
drwxrwx---.  2 grid oinstall   6 Mar 20 11:51 audit

chmod 770 $ORACLE_BASE/cfgtoollogs/dbca

5. Run cluvfy could to check pre-requisite for RAC DB (output is without verbose option).

cluvfy stage -pre dbcfg -allnodes -d /opt/app/oracle/product/18.x.0/dbhome_1

Verifying Physical Memory ...PASSED
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...PASSED
Verifying Free Space: rhel72:/tmp ...PASSED
Verifying Free Space: rhel71:/tmp ...PASSED
Verifying User Existence: oracle ...
  Verifying Users With Same UID: 501 ...PASSED
Verifying User Existence: oracle ...PASSED
Verifying Group Existence: dgdba ...PASSED
Verifying Group Existence: dba ...PASSED
Verifying Group Existence: racdba ...PASSED
Verifying Group Existence: backupdba ...PASSED
Verifying Group Existence: oper ...PASSED
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: backupdba ...PASSED
Verifying Group Membership: racdba ...PASSED
Verifying Group Membership: dgdba ...PASSED
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Group Membership: dba ...PASSED
Verifying Group Membership: oper ...PASSED
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying Package: binutils-2.23.52.0.1 ...PASSED
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: libgcc-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...PASSED
Verifying Package: sysstat-10.1.5 ...PASSED
Verifying Package: gcc-c++-4.8.2 ...PASSED
Verifying Package: ksh ...PASSED
Verifying Package: make-3.82 ...PASSED
Verifying Package: glibc-2.17 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.17 (x86_64) ...PASSED
Verifying Package: libaio-0.3.109 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.109 (x86_64) ...PASSED
Verifying Package: smartmontools-6.2-4 ...PASSED
Verifying Package: net-tools-2.0-0.17 ...PASSED
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...PASSED
Verifying Package: libxcb-1.11 (x86_64) ...PASSED
Verifying Package: libX11-1.6.3 (x86_64) ...PASSED
Verifying Package: libXau-1.0.8 (x86_64) ...PASSED
Verifying Package: libXi-1.7.4 (x86_64) ...PASSED
Verifying Package: libXtst-1.2.2 (x86_64) ...PASSED
Verifying Current Group ID ...PASSED
Verifying CRS Integrity ...
  Verifying Clusterware Version Consistency ...PASSED
Verifying CRS Integrity ...PASSED
Verifying Node Application Existence ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Single Client Access Name (SCAN) ...
  Verifying DNS/NIS name service 'rac-scan.domain.net' ...
    Verifying Name Service Switch Configuration File Integrity ...PASSED
  Verifying DNS/NIS name service 'rac-scan.domain.net' ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verifying ASM Integrity ...
  Verifying Node Connectivity ...
    Verifying Hosts File ...PASSED
    Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
    Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
    Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
  Verifying Node Connectivity ...PASSED
Verifying ASM Integrity ...PASSED
Verifying Database Clusterware Version Compatibility ...PASSED
Verifying File system mount options for path ORACLE_HOME ...PASSED
Verifying /dev/shm mounted as temporary file system ...FAILED (PRVE-0421)

Pre-check for database configuration was unsuccessful on all the nodes.


Failures were encountered during execution of CVU verification request "stage -pre dbcfg".

Verifying /dev/shm mounted as temporary file system ...FAILED
rhel72: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

rhel71: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

6. Create the database using dbca. 7. Create PDBs are required and associate services with them so PDBs are auto started. Assuming PDB named pdb18c is created, add a database service associated with PDB with

srvctl add service -db ent18c -pdb pdb18c -service pdbsrv -preferred "ent18c1,ent18c2"
srvctl start service -db ent18c -s pdbsrv

8. Finally, the stopping of the ASM proxy instances has no relevance to the "flex" functioning of DB instances. If an ASM instances in one node goes down, these DB instances are able to use a remote ASM instance to access the disks. This could be verified with the following.

[grid@rhel71 ~]$  srvctl status asm -detail
ASM is running on rhel71,rhel72
ASM is enabled.
ASM instance +ASM1 is running on node rhel71
Number of connected clients: 2
Client names: ent18c1:ent18c:rhel7-cluster rhel71.domain.net:_OCR:rhel7-cluster
ASM instance +ASM2 is running on node rhel72
Number of connected clients: 3
Client names: -MGMTDB:_mgmtdb:rhel7-cluster ent18c2:ent18c:rhel7-cluster rhel72.domain.net:_OCR:rhel7-cluster

The above output shows the clients connected to each ASM instance on each node. Currently ASM1 has 2 clients while ASM2 has 3 clients, *OCR is available as client on both ASM instances.
Then an ASM instance is stopped.

srvctl stop asm  -node rhel72  -force

When one of the ASM instances is stopped forcefully those clients start using the service of the available ASM instance

srvctl status asm -detail
ASM is running on rhel71
ASM is enabled.
ASM instance +ASM1 is running on node rhel71
Number of connected clients: 4
Client names: -MGMTDB:_mgmtdb:rhel7-cluster ent18c1:ent18c:rhel7-cluster ent18c2:ent18c:rhel7-cluster rhel71.domain.net:_OCR:rhel7-cluster

As seen above, clients that previously clients of ASM2 instance are now using ASM1 instance to access ASM disks. (total client count is 4 not 5 due to not duplicating the OCR* client).
When ASM instance is restarted the clients automatically select the local ASM instance over remote ASM instances.

srvctl start asm  -node rhel72
srvctl status asm -detail
ASM is running on rhel71,rhel72
ASM is enabled.
ASM instance +ASM1 is running on node rhel71
Number of connected clients: 2
Client names: ent18c1:ent18c:rhel7-cluster rhel71.domain.net:_OCR:rhel7-cluster
ASM instance +ASM2 is running on node rhel72
Number of connected clients: 3
Client names: -MGMTDB:_mgmtdb:rhel7-cluster ent18c2:ent18c:rhel7-cluster rhel72.domain.net:_OCR:rhel7-cluster

This concludes the setting up RAC DB on 18c.

Related Posts
Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Database
Installing 12c (12.1.0.1) RAC on RHEL 6 with Role Separation - Database Software
Installing 12c (12.1.0.1) RAC on RHEL 6 with Role Separation - Creating CDB & PDB