Removing a Failed Node From the Cluster

July 14, 2016, 12:00 am

≫ Next: GUID Directory Not Created for PDBs During Full Restore in 12.1.0.2

≪ Previous: Moving Role Separated Oracle Restart Setup to Single User Setup

This post list steps for removing a failed node from a cluster. The steps differs from steps in the previous node deletion posts (11gR1,11gR2 and 12c) such that one node has suffered a catastrophic failure and is not available for any kind of command or script executions. Therefore all the activities involved in removing the failed node are executed from a surviving node.
The environment used in this case is a two node RAC with role separation(11.2.0.4). Under normal operation it has the following resources and status. (formatted status)

Resource Name                       Type                      Target             State              Host
-------------                       ------                    -------            --------           ----------
ora.CLUSTER_DG.dg                   ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.CLUSTER_DG.dg                   ora.diskgroup.type        ONLINE             ONLINE             rhel6m2
ora.DATA.dg                         ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.DATA.dg                         ora.diskgroup.type        ONLINE             ONLINE             rhel6m2
ora.FLASH.dg                        ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.FLASH.dg                        ora.diskgroup.type        ONLINE             ONLINE             rhel6m2
ora.MYLISTENER.lsnr                 ora.listener.type         ONLINE             ONLINE             rhel6m1
ora.MYLISTENER.lsnr                 ora.listener.type         ONLINE             ONLINE             rhel6m2
ora.MYLISTENER_SCAN1.lsnr           ora.scan_listener.type    ONLINE             ONLINE             rhel6m2
ora.asm                             ora.asm.type              ONLINE             ONLINE             rhel6m1
ora.asm                             ora.asm.type              ONLINE             ONLINE             rhel6m2
ora.cvu                             ora.cvu.type              ONLINE             ONLINE             rhel6m2
ora.gsd                             ora.gsd.type              OFFLINE            OFFLINE
ora.gsd                             ora.gsd.type              OFFLINE            OFFLINE
ora.net1.network                    ora.network.type          ONLINE             ONLINE             rhel6m1
ora.net1.network                    ora.network.type          ONLINE             ONLINE             rhel6m2
ora.oc4j                            ora.oc4j.type             ONLINE             ONLINE             rhel6m2
ora.ons                             ora.ons.type              ONLINE             ONLINE             rhel6m1
ora.ons                             ora.ons.type              ONLINE             ONLINE             rhel6m2
ora.registry.acfs                   ora.registry.acfs.type    ONLINE             ONLINE             rhel6m1
ora.registry.acfs                   ora.registry.acfs.type    ONLINE             ONLINE             rhel6m2
ora.rhel6m1.vip                     ora.cluster_vip_net1.type ONLINE             ONLINE             rhel6m1
ora.rhel6m2.vip                     ora.cluster_vip_net1.type ONLINE             ONLINE             rhel6m2
ora.scan1.vip                       ora.scan_vip.type         ONLINE             ONLINE             rhel6m2
ora.std11g2.db                      ora.database.type         ONLINE             ONLINE             rhel6m1
ora.std11g2.db                      ora.database.type         ONLINE             ONLINE             rhel6m2
ora.std11g2.myservice.svc           ora.service.type          ONLINE             ONLINE             rhel6m1
ora.std11g2.myservice.svc           ora.service.type          ONLINE             ONLINE             rhel6m2
ora.std11g2.abx.domain.net.svc      ora.service.type          ONLINE             ONLINE             rhel6m2
ora.std11g2.abx.domain.net.svc      ora.service.type          ONLINE             ONLINE             rhel6m1

After the node 2 (rhel6m2 node in this case) suffers a catastrophic failure, resources and status is as below. There are offline and failed over (vip) resources from rhel6m2.

Resource Name                       Type                      Target             State              Host
-------------                       ------                    -------            --------           ----------
ora.CLUSTER_DG.dg                   ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.DATA.dg                         ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.FLASH.dg                        ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.MYLISTENER.lsnr                 ora.listener.type         ONLINE             ONLINE             rhel6m1
ora.MYLISTENER_SCAN1.lsnr           ora.scan_listener.type    ONLINE             ONLINE             rhel6m1
ora.asm                             ora.asm.type              ONLINE             ONLINE             rhel6m1
ora.cvu                             ora.cvu.type              ONLINE             ONLINE             rhel6m1
ora.gsd                             ora.gsd.type              OFFLINE            OFFLINE
ora.net1.network                    ora.network.type          ONLINE             ONLINE             rhel6m1
ora.oc4j                            ora.oc4j.type             ONLINE             ONLINE             rhel6m1
ora.ons                             ora.ons.type              ONLINE             ONLINE             rhel6m1
ora.registry.acfs                   ora.registry.acfs.type    ONLINE             ONLINE             rhel6m1
ora.rhel6m1.vip                     ora.cluster_vip_net1.type ONLINE             ONLINE             rhel6m1
ora.rhel6m2.vip                     ora.cluster_vip_net1.type ONLINE             INTERMEDIATE       rhel6m1
ora.scan1.vip                       ora.scan_vip.type         ONLINE             ONLINE             rhel6m1
ora.std11g2.db                      ora.database.type         ONLINE             ONLINE             rhel6m1
ora.std11g2.db                      ora.database.type         ONLINE             OFFLINE
ora.std11g2.myservice.svc           ora.service.type          ONLINE             ONLINE             rhel6m1
ora.std11g2.myservice.svc           ora.service.type          ONLINE             OFFLINE
ora.std11g2.abx.domain.net.svc      ora.service.type          ONLINE             OFFLINE
ora.std11g2.abx.domain.net.svc      ora.service.type          ONLINE             ONLINE             rhel6m1

Removing of resources of the failed node begins at database resource level. There are two services running and they both have the DB instance on the failed node as a preferred instance (output is condensed)

srvctl config service -d std11g2
Service name: myservice
Service is enabled
Server pool: std11g2_myservice
Cardinality: 2
...
Preferred instances: std11g21,std11g22
Available instances:
Service name: abx.domain.net
Service is enabled
Server pool: std11g2_abx.domain.net
Cardinality: 2
...
Preferred instances: std11g21,std11g22
Available instances:

Modify the service configuration so that only the surviving instances are set as preferred instances.

$ srvctl modify service -s myservice -d std11g2 -n -i std11g21 -f
$ srvctl modify service -s abx.domain.net -d std11g2 -n -i std11g21 -f

$ srvctl config service -d std11g2
Service name: myservice
Service is enabled
Server pool: std11g2_myservice
Cardinality: 1
..
Preferred instances: std11g21
Available instances:
Service name: abx.domain.net
Service is enabled
Server pool: std11g2_abx.domain.net
Cardinality: 1
..
Preferred instances: std11g21
Available instances:

$ srvctl status service -d std11g2
Service myservice is running on instance(s) std11g21
Service abx.domain.net is running on instance(s) std11g21

Remove the database instance on the failed node

srvctl config database -d std11g2
Database unique name: std11g2
Database name: std11g2
...
Database instances: std11g21,std11g22
Disk Groups: DATA,FLASH
Mount point paths:
Services: myservice,abx.domain.net
Type: RAC
Database is administrator managed

This is done using DBCA's instance management option. If the listener has a non-default name and port then accessing the DB will fail with below message.

To fix this create a default listener (name listener and port 1521). Also if VNCR is used then remove the failed node from the registration list. Proceed to instance deletion by selecting the inactive instance on the failed node.

As node 2 is not available following warning will be issued. Click continue and proceed. During the execution various other warning will appear such as unable to remove /etc/oratab etc all of these could be ignored.

However DBCA didn't run till end, at 67% (observed through repeated runs on this 11.2.0.4 environment) following dialog box appeared. As seen on the screenshot it has no message, just an OK button. Clicking it doesn't end the DBCA session but goes to the beginning and exit the DBCA clicking cancel afterwards.

However this doesn't appear to be a failure on the DBCA to remove the instance. In fact instance is removed as subsequent instance operation only list the instance on the surviving node.

Querying the database also shows that instance 2 (std11g22 in this case) related undo tablespace and redo logs have been removed and only surviving instance related undo tablespace and redo logs are available.

SQL> select name from v$tablespace;

NAME
------------------------------
SYSTEM
SYSAUX
UNDOTBS1
TEMP
USERS
EXAMPLE
TEST

7 rows selected.

SQL> select * from v$log;

    GROUP#    THREAD#  SEQUENCE#      BYTES  BLOCKSIZE    MEMBERS ARC STATUS           FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# NEXT_TIME
---------- ---------- ---------- ---------- ---------- ---------- --- ---------------- ------------- --------- ------------ ---------
         1          1       1598   52428800        512          2 NO  CURRENT               68471125 07-JUL-16   2.8147E+14
         2          1       1597   52428800        512          2 YES INACTIVE              68467762 07-JUL-16     68471125 07-JUL-16

srvctl config database -d std11g2
Database unique name: std11g2
Database name: std11g2
...
Database instances: std11g21
Disk Groups: DATA,FLASH
Mount point paths:
Services: myservice,abx.domain.net
Type: RAC
Database is administrator managed

Once the database resources are removed next step is to remove the Oracle database home entry for the failed node from the inventory.

As the node is unavailable, there's no un-installation involved. Run the inventory update command with surviving nodes. Inventory content for the Oracle home before the failed node is removed.

<HOME NAME="OraDb11g_home2" LOC="/opt/app/oracle/product/11.2.0/dbhome_4" TYPE="O" IDX="4">
<NODE_LIST>
<NODE NAME="rhel6m1"/>
<NODE NAME="rhel6m2"/>
</NODE_LIST>
</HOME>

After the inventory update

 ./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={rhel6m1}"
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 4095 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /opt/app/oraInventory
'UpdateNodeList' was successful.

<HOME NAME="OraDb11g_home2" LOC="/opt/app/oracle/product/11.2.0/dbhome_4" TYPE="O" IDX="4">
<NODE_LIST>
<NODE NAME="rhel6m1"/>
</NODE_LIST>
</HOME>

Next step is to remove the cluster resources and the node itself. If any of the node is in pin stat, unpin them. In this case both nodes are unpinned

olsnodes -s -t
rhel6m1 Active  Unpinned
rhel6m2 Inactive        Unpinned

Stop and remove the VIP resource of the failed node

# srvctl stop  vip -i rhel6m2-vip -f
# srvctl remove vip -i rhel6m2-vip -f

Remove the failed node from the cluster configuration

#  crsctl delete node -n rhel6m2
CRS-4661: Node rhel6m2 successfully deleted.

Finally remove the grid home for the failed node from the inventory. Before inventory update

<HOME NAME="Ora11g_gridinfrahome2" LOC="/opt/app/11.2.0/grid4" TYPE="O" IDX="3" CRS="true">
<NODE_LIST>
<NODE NAME="rhel6m1"/>
<NODE NAME="rhel6m2"/>
</NODE_LIST>
</HOME>

After inventory update

./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={rhel6m1}" CRS=TRUE
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 4095 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /opt/app/oraInventory
'UpdateNodeList' was successful.

<HOME NAME="Ora11g_gridinfrahome2" LOC="/opt/app/11.2.0/grid4" TYPE="O" IDX="3" CRS="true">
<NODE_LIST>
<NODE NAME="rhel6m1"/>
</NODE_LIST>
</HOME>

Validate the node removal with cluvfy

cluvfy stage -post  nodedel -n rhel6m2

Performing post-checks for node removal
Checking CRS integrity...
Clusterware version consistency passed
CRS integrity check passed
Node removal check passed
Post-check for node removal was successful.

Remove the default listener if one was created during instance remove step. The final status of resource is as below.

Resource Name                       Type                      Target             State              Host
-------------                       ------                    -------            --------           ----------
ora.CLUSTER_DG.dg                   ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.DATA.dg                         ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.FLASH.dg                        ora.diskgroup.type        ONLINE             ONLINE             rhel6m1
ora.MYLISTENER.lsnr                 ora.listener.type         ONLINE             ONLINE             rhel6m1
ora.MYLISTENER_SCAN1.lsnr           ora.scan_listener.type    ONLINE             ONLINE             rhel6m1
ora.asm                             ora.asm.type              ONLINE             ONLINE             rhel6m1
ora.cvu                             ora.cvu.type              ONLINE             ONLINE             rhel6m1
ora.gsd                             ora.gsd.type              OFFLINE            OFFLINE
ora.net1.network                    ora.network.type          ONLINE             ONLINE             rhel6m1
ora.oc4j                            ora.oc4j.type             ONLINE             ONLINE             rhel6m1
ora.ons                             ora.ons.type              ONLINE             ONLINE             rhel6m1
ora.registry.acfs                   ora.registry.acfs.type    ONLINE             ONLINE             rhel6m1
ora.rhel6m1.vip                     ora.cluster_vip_net1.type ONLINE             ONLINE             rhel6m1
ora.scan1.vip                       ora.scan_vip.type         ONLINE             ONLINE             rhel6m1
ora.std11g2.db                      ora.database.type         ONLINE             ONLINE             rhel6m1
ora.std11g2.myservice.svc           ora.service.type          ONLINE             ONLINE             rhel6m1
ora.std11g2.abx.domain.net.svc      ora.service.type          ONLINE             ONLINE             rhel6m1

Useful metalink notes
How to remove/delete a node from Grid Infrastructure Clusterware when the node has failed [ID 1262925.1]
Steps to Remove Node from Cluster When the Node Crashes Due to OS/Hardware Failure and cannot boot up [ID 466975.1]
RAC on Windows: How to Remove a Node from a Cluster When the Node Crashes Due to OS/Hardware Failure and Cannot Boot [ID 832054.1]

Related Post
Deleting a Node From 12cR1 RAC
Deleting a Node From 11gR2 RAC
Deleting a 11gR1 RAC Node

↧

GUID Directory Not Created for PDBs During Full Restore in 12.1.0.2

August 1, 2016, 12:00 am

≫ Next: Resetlogs Fails with ORA-00349

≪ Previous: Removing a Failed Node From the Cluster

Each database (non-CDB, CDB, PDB) on 12c has a generic unique identifier (GUID) associated with it. This GUID will not change throughout the life of the container/non-CDB. The 12c Admin guide says "the PDB GUID is primarily used to generate names for directories that store the PDB’s files, including both Oracle Managed Files directories and non-Oracle Managed Files directories". But it seems during RMAN restore the PDB related data files are not restored into a directory structure with GUID. There is a variant of this issue where "RMAN restore does not create GUID directory for pre-12c database" (16874123.8) which is fixed on 12.1.0.2 according to 1683802.1. The version used for testing is 12.1.0.2, so not sure if what's observed is a new bug or a different manifestation of the bug 16874123.
The source database has 3 PDBs (not counting the seed PDB) and data files are contained within a GUID directory.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDBONE                         READ WRITE NO
         4 PDBTWO                         MOUNTED
         5 PDBTHREE                       MOUNTED

SQL> select con_id,name from v$datafile order by 1;

    CON_ID NAME
---------- ----------------------------------------------------------------------------------------------------
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_bocndoww_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_rootbs_bq3ngj79_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_users_bocng186_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_undotbs1_bocnf2j0_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_bocndyz0_.dbf
         2 /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/o1_mf_system_bocnds90_.dbf
         2 /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/o1_mf_sysaux_bocnf0mr_.dbf
         3 /opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_sysaux_bqc9rzvz_.dbf
         3 /opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_system_bqc9rzvt_.dbf
         3 /opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_test_bqc9yss9_.dbf
         4 /opt/app/oracle/oradata/CDB12C/1803968083DD030DE0535500A8C08A81/datafile/o1_mf_sysaux_bqcb6642_.dbf
         4 /opt/app/oracle/oradata/CDB12C/1803968083DD030DE0535500A8C08A81/datafile/o1_mf_system_bqcb663x_.dbf
         5 /opt/app/oracle/oradata/CDB12C/3643E46698E543F4E0535D00A8C0BDF3/datafile/o1_mf_system_cq2d54cz_.dbf
         5 /opt/app/oracle/oradata/CDB12C/3643E46698E543F4E0535D00A8C0BDF3/datafile/o1_mf_sysaux_cq2d54dx_.dbf

A full backup of the database is taken and restored in another server which is identical to the source DB (OS, Oracle version including patch level etc). db_create_file_dest parameter is same on both source and restored DBs pfiles.
The restore output is shown below which indicates that PDB related files are restored into a directory structure with GUID.

RMAN> restore controlfile from '/home/oracle/backup/control.ctl';

Starting restore at 28-JUL-16
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=243 device type=DISK

channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
output file name=/opt/app/oracle/oradata/CDB12C/controlfile/o1_mf_csn2c773_.ctl
output file name=/opt/app/oracle/fast_recovery_area/CDB12C/controlfile/o1_mf_csn2c7hf_.ctl
Finished restore at 28-JUL-16

RMAN> alter database mount;

Statement processed
released channel: ORA_DISK_1

RMAN> run {
2> restore database;
3> recover database;
4> }

Starting restore at 28-JUL-16
Starting implicit crosscheck backup at 28-JUL-16
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=243 device type=DISK
Crosschecked 8 objects
Finished implicit crosscheck backup at 28-JUL-16

Starting implicit crosscheck copy at 28-JUL-16
using channel ORA_DISK_1
Crosschecked 2 objects
Finished implicit crosscheck copy at 28-JUL-16

searching for all files in the recovery area
cataloging files...
no files cataloged

using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_bocndoww_.dbf
channel ORA_DISK_1: restoring datafile 00003 to /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_bocndyz0_.dbf
channel ORA_DISK_1: restoring datafile 00005 to /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_undotbs1_bocnf2j0_.dbf
channel ORA_DISK_1: restoring datafile 00006 to /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_users_bocng186_.dbf
channel ORA_DISK_1: restoring datafile 00022 to /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_rootbs_bq3ngj79_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/bkp5brboipc_1_1
channel ORA_DISK_1: piece handle=/home/oracle/backup/bkp5brboipc_1_1 tag=TAG20160727T140115
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:56
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00025 to /opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_system_bqc9rzvt_.dbf
channel ORA_DISK_1: restoring datafile 00026 to /opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_sysaux_bqc9rzvz_.dbf
channel ORA_DISK_1: restoring datafile 00027 to /opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_test_bqc9yss9_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/bkp5crboir3_1_1
channel ORA_DISK_1: piece handle=/home/oracle/backup/bkp5crboir3_1_1 tag=TAG20160727T140115
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:18
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00028 to /opt/app/oracle/oradata/CDB12C/1803968083DD030DE0535500A8C08A81/datafile/o1_mf_system_bqcb663x_.dbf
channel ORA_DISK_1: restoring datafile 00029 to /opt/app/oracle/oradata/CDB12C/1803968083DD030DE0535500A8C08A81/datafile/o1_mf_sysaux_bqcb6642_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/bkp5drboirj_1_1
channel ORA_DISK_1: piece handle=/home/oracle/backup/bkp5drboirj_1_1 tag=TAG20160727T140115
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:25
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00039 to /opt/app/oracle/oradata/CDB12C/3643E46698E543F4E0535D00A8C0BDF3/datafile/o1_mf_system_cq2d54cz_.dbf
channel ORA_DISK_1: restoring datafile 00040 to /opt/app/oracle/oradata/CDB12C/3643E46698E543F4E0535D00A8C0BDF3/datafile/o1_mf_sysaux_cq2d54dx_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/bkp5erboisf_1_1
channel ORA_DISK_1: piece handle=/home/oracle/backup/bkp5erboisf_1_1 tag=TAG20160727T140115
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00002 to /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/o1_mf_system_bocnds90_.dbf
channel ORA_DISK_1: restoring datafile 00004 to /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/o1_mf_sysaux_bocnf0mr_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/bkp5frboisv_1_1
channel ORA_DISK_1: piece handle=/home/oracle/backup/bkp5frboisv_1_1 tag=TAG20160727T140115
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:25
Finished restore at 28-JUL-16

Starting recover at 28-JUL-16
using channel ORA_DISK_1

starting media recovery

channel ORA_DISK_1: starting archived log restore to default destination
channel ORA_DISK_1: restoring archived log
archived log thread=1 sequence=221
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/arch5grboitr_1_1
channel ORA_DISK_1: piece handle=/home/oracle/backup/arch5grboitr_1_1 tag=TAG20160727T140338
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
archived log file name=/opt/app/oracle/fast_recovery_area/CDB12C/archivelog/2016_07_28/o1_mf_1_221_csn2jcf4_.arc thread=1 sequence=221
channel default: deleting archived log(s)
archived log file name=/opt/app/oracle/fast_recovery_area/CDB12C/archivelog/2016_07_28/o1_mf_1_221_csn2jcf4_.arc RECID=518 STAMP=918398107
unable to find archived log
archived log thread=1 sequence=222
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 07/28/2016 14:35:18
RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 222 and starting SCN of 3544373

Running report schema shows that all the datafiles for PDBs are created at the same level as the CDB datafiles except for temp files used by PDBs which are listed with a directory path containing GUID.


RMAN> report schema;

RMAN-06139: WARNING: control file is not current for REPORT SCHEMA
Report of database schema for database with db_unique_name CDB12C

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    810      SYSTEM               ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2cvqv_.dbf
2    260      PDB$SEED:SYSTEM      ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2hg6q_.dbf
3    980      SYSAUX               ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2cvqo_.dbf
4    595      PDB$SEED:SYSAUX      ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2hg6g_.dbf
5    835      UNDOTBS1             ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_undotbs1_csn2cvqr_.dbf
6    5        USERS                ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_users_csn2cvr5_.dbf
22   10       ROOTBS               ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_rootbs_csn2cvr2_.dbf
25   270      PDBONE:SYSTEM        ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2fpvd_.dbf
26   615      PDBONE:SYSAUX        ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2fpv4_.dbf
27   10       PDBONE:TEST          ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_test_csn2fpvh_.dbf
28   270      PDBTWO:SYSTEM        ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2g9g0_.dbf
29   605      PDBTWO:SYSAUX        ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2g6mg_.dbf
39   260      PDBTHREE:SYSTEM      ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2gz2w_.dbf
40   605      PDBTHREE:SYSAUX      ***     /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2gz2c_.dbf

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    20       TEMP                 32767       /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_temp_bocnf32d_.tmp
2    20       PDB$SEED:TEMP        32767       /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/o1_mf_temp_bocnf33b_.tmp
3    20       PDBONE:TEMP          32767       /opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_temp_bqc9rzw2_.dbf
4    20       PDBTWO:TEMP          32767       /opt/app/oracle/oradata/CDB12C/1803968083DD030DE0535500A8C08A81/datafile/o1_mf_temp_bqcb6645_.dbf
5    20       PDBTHREE:TEMP        32767       /opt/app/oracle/oradata/CDB12C/3643E46698E543F4E0535D00A8C0BDF3/datafile/o1_mf_temp_cq2d54f1_.dbf

However these temp files or the GUID directory doesn't exists on the filesystem.

$ ls -l /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/
ls: cannot access /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/: No such file or directory

$ ls -l /opt/app/oracle/oradata/CDB12C/
total 8
drwxr-x--- 2 oracle oinstall 4096 Jul 28 14:32 controlfile
drwxr-x--- 2 oracle oinstall 4096 Jul 28 14:34 datafile

Opening the database in reset log mode, will dynamically create the GUID directories and temp files inside them

RMAN> alter database open resetlogs;

Statement processed

$ ls -l /opt/app/oracle/oradata/CDB12C/
total 28
drwxr-x--- 3 oracle oinstall 4096 Jul 28 14:37 161DBD44EAAA7AC0E0535500A8C01F27
drwxr-x--- 3 oracle oinstall 4096 Jul 28 14:37 18037D5BA62802CEE0535500A8C0EC8F
drwxr-x--- 3 oracle oinstall 4096 Jul 28 14:37 1803968083DD030DE0535500A8C08A81
drwxr-x--- 3 oracle oinstall 4096 Jul 28 14:37 3643E46698E543F4E0535D00A8C0BDF3
drwxr-x--- 2 oracle oinstall 4096 Jul 28 14:32 controlfile
drwxr-x--- 2 oracle oinstall 4096 Jul 28 14:37 datafile
drwxr-x--- 2 oracle oinstall 4096 Jul 28 14:37 onlinelog

$ ls -l /opt/app/oracle/oradata/CDB12C/161DBD44EAAA7AC0E0535500A8C01F27/datafile/
total 1024
-rw-r----- 1 oracle oinstall 20979712 Jul 28 14:37 o1_mf_temp_csn2n97z_.tmp

However the PDBs' datafiles are all in the same OMF path

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDBONE                         READ WRITE NO
         4 PDBTWO                         MOUNTED
         5 PDBTHREE                       MOUNTED
SQL>  select con_id,name from v$datafile order by 1;

    CON_ID NAME
---------- ---------------------------------------------------------------------
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2cvqv_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_rootbs_csn2cvr2_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_users_csn2cvr5_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_undotbs1_csn2cvqr_.dbf
         1 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2cvqo_.dbf
         2 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2hg6q_.dbf
         2 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2hg6g_.dbf
         3 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2fpv4_.dbf
         3 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2fpvd_.dbf
         3 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_test_csn2fpvh_.dbf
         4 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2g6mg_.dbf
         4 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2g9g0_.dbf
         5 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2gz2w_.dbf
         5 /opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2gz2c_.dbf

If a new tablespace or data file is added to one of the PDBs this will be created in a directory path with GUID.

SQL> alter session set container=pdbone;

SQL>  show con_name

CON_NAME
------------------------------
PDBONE

SQL> select name from v$datafile;

NAME
----------------------------------------------------------------------------------------------------
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_undotbs1_csn2cvqr_.dbf
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2fpvd_.dbf
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2fpv4_.dbf
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_test_csn2fpvh_.dbf

SQL> create tablespace newtbs DATAFILE SIZE 10M;

Tablespace created.

SQL>  select name from v$datafile;

NAME
----------------------------------------------------------------------------------------------------
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_undotbs1_csn2cvqr_.dbf
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_system_csn2fpvd_.dbf
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_sysaux_csn2fpv4_.dbf
/opt/app/oracle/oradata/CDB12C/datafile/o1_mf_test_csn2fpvh_.dbf
/opt/app/oracle/oradata/CDB12C/18037D5BA62802CEE0535500A8C0EC8F/datafile/o1_mf_newtbs_csn2s0yz_.dbf

The restore was also tested by pre-creating the directory path with GUID, however there was no difference in the outcome. Also tested with latest PSU (July 2016) applied on source DB's home and new servers oracle home, and using a new backup taken from source DB after PSU applied. No difference in that case either. Also tested with patch 20464614 applied (mentioned in 1576755.1 in relation to OMF/ASM and PDBs). But didn't observe any difference in the final outcome.
A service request has been raised.

Useful metalink notes
Bug 16874123 - RMAN restore does not create GUID directory for pre-12c database [ID 16874123.8]
Step by Step Examples of Migrating non-CDBs and PDBs Using ASM for File Storage [ID 1576755.1]

↧

Resetlogs Fails with ORA-00349

August 14, 2016, 12:00 am

≫ Next: Move a PDB Between Servers

≪ Previous: GUID Directory Not Created for PDBs During Full Restore in 12.1.0.2

During a database restore on a separate server from the source DB following error occurred during open resetlogs.

RMAN> alter database open resetlogs;

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of sql statement command at 08/10/2016 18:48:18
ORA-00349: failure obtaining block size for '/data/oradata/apcdb/redo01.log'
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 9

The source database was on a non-OMF setup and the DB was restored on OMF paths. As such the original redo directory paths does not exists on the restored sever. Typical solution is to clear the unarchived log file groups but option failed in this case.

SQL> alter database clear unarchived logfile group 1 ;
alter database clear unarchived logfile group 1
*
ERROR at line 1:
ORA-00349: failure obtaining block size for '/data/oradata/apcdb/redo01.log'
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 9

However it is possible to drop and add log file groups

SQL> alter database drop logfile group 1;
Database altered.

SQL> alter database add logfile group 1;
Database altered.

SQL>  alter database drop logfile group 3;
Database altered.

SQL> alter database add logfile group 3;
Database altered.

Yet was unable to drop and add one log group as it was considered current

SQL> alter database drop logfile group 2;
alter database drop logfile group 2
*
ERROR at line 1:
ORA-01623: log 2 is current log for instance apcdb (thread 1) - cannot drop
ORA-00312: online log 2 thread 1: '/data/oradata/apcdb/redo02.log'

SQL> select group#,status from v$log;

    GROUP# STATUS
---------- ----------------
         1 UNUSED
         3 UNUSED
2 CLEARING_CURRENT

Solution in this case is to backup the control file to trace and edit the log file locations.

SQL> ALTER DATABASE BACKUP CONTROLFILE TO TRACE AS '/tmp/control.sql' resetlogs ;

On the trace file the control file create was as follows

CREATE CONTROLFILE REUSE DATABASE "APCDB" RESETLOGS  ARCHIVELOG
    MAXLOGFILES 16
    MAXLOGMEMBERS 3
    MAXDATAFILES 1024
    MAXINSTANCES 8
    MAXLOGHISTORY 292
LOGFILE
  GROUP 1 (
'/opt/app/oracle/oradata/APCDB/onlinelog/o1_mf_1_ctptbqxp_.log',
'/opt/app/oracle/flash_recovery_area/APCDB/onlinelog/o1_mf_1_ctptbr4t_.log'
  ) SIZE 100M BLOCKSIZE 512,
 GROUP 2 '/data/oradata/apcdb/redo02.log'  SIZE 50M BLOCKSIZE 512,
  GROUP 3 (
'/opt/app/oracle/oradata/APCDB/onlinelog/o1_mf_3_ctptc5mn_.log',
'/opt/app/oracle/flash_recovery_area/APCDB/onlinelog/o1_mf_3_ctptc5tb_.log'
  ) SIZE 100M BLOCKSIZE 512
-- STANDBY LOGFILE
DATAFILE
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_system_ctpskmcr_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_sysaux_ctpskmcd_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_undotbs1_ctpskmgg_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_system_ctpsnlq7_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_users_ctpskmhd_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_sysaux_ctpsnlp4_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_system_ctpsm0jd_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_sysaux_ctpsm0hp_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_system_ctpsmsmb_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_sysaux_ctpsmsm4_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_test_ctpsm0l1_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_test_ctpsmsnt_.dbf',
'/opt/app/oracle/oradata/APCDB/datafile/o1_mf_newtest_ctpsm0m8_.dbf'
CHARACTER SET AL32UTF8
;

As seen on the trace the log file groups 1 and 3 are using the new OMF paths but group 2 still refer to source DB path where the backups were taken from. Two options are available to deal with the group 2, one is to replace the group 2 with an OMF path, in this case

GROUP 2 '/data/oradata/apcdb/redo02.log'  SIZE 50M BLOCKSIZE 512,

with

GROUP 2 (
'/opt/app/oracle/oradata/APCDB/onlinelog/o1_mf_2_captbqxp_.log',
'/opt/app/oracle/flash_recovery_area/APCDB/onlinelog/o1_mf_2_captbr4t_.log'
  ),

Or the second option is to completely omit the group 2 and then add group 2 once the control file is created and before resetlogs is run again. Once the control file create section is edited start the database in nomount and run the script

SQL> startup nomount;
SQL> @/tmp/control.sql

This will leave the DB on mount mode. If group 2 was omitted from the control file create then add the group 2 now.

SQL> select * from v$log;

    GROUP#    THREAD#  SEQUENCE#      BYTES  BLOCKSIZE    MEMBERS ARC STATUS           FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# NEXT_TIME     CON_ID
---------- ---------- ---------- ---------- ---------- ---------- --- ---------------- ------------- --------- ------------ --------- ----------
         3          1          0  104857600        512          2 YES CURRENT                      0                      0                    0
         1          1          0  104857600        512          2 YES UNUSED                       0                      0                    0

SQL> alter database add logfile group 2;
Database altered.

SQL> select * from v$log;

    GROUP#    THREAD#  SEQUENCE#      BYTES  BLOCKSIZE    MEMBERS ARC STATUS           FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# NEXT_TIME     CON_ID
---------- ---------- ---------- ---------- ---------- ---------- --- ---------------- ------------- --------- ------------ --------- ----------
         1          1          0  104857600        512          2 YES UNUSED                       0                      0                    0
         3          1          0  104857600        512          2 YES CURRENT                      0                      0                    0
         2          1          0  104857600        512          2 YES UNUSED                       0                      0                    0

Run a recovery using backup control file until cancel.


SQL> RECOVER DATABASE USING BACKUP CONTROLFILE UNTIL CANCEL
ORA-00279: change 4285165 generated at 08/10/2016 15:52:55 needed for thread 1
ORA-00289: suggestion :
/opt/app/oracle/flash_recovery_area/APCDB/archivelog/2016_08_10/o1_mf_1_206_%u_.
arc
ORA-00280: change 4285165 for thread 1 is in sequence #206

Specify log: {=suggested | filename | AUTO | CANCEL}
cancel
Media recovery cancelled.

Finally run the open resetlogs

SQL> alter database open resetlogs;
Database altered.

Useful metalink note
ALTER DATABASE OPEN RESETLOGS fails with ORA-00392 [ID 1352133.1]

↧

Move a PDB Between Servers

September 1, 2016, 12:00 am

≫ Next: Plugging a SE2 non-CDB as an EE PDB Using File Copying and Remote Link

≪ Previous: Resetlogs Fails with ORA-00349

This post list the steps for moving a PDB from it's current CDB to another CDB on a different server. The detonation CDB and the PDB are patched to the same level.
Destination CDB patches

SQL> select con_id,patch_id,action,status,description from cdb_registry_sqlpatch where con_id = 1 order by action_time;

    CON_ID   PATCH_ID ACTION          STATUS          DESCRIPTION
---------- ---------- --------------- --------------- ----------------------------------------------------------------------------------------------------
         1   21555660 APPLY           SUCCESS         Database PSU 12.1.0.2.5, Oracle JavaVM Component (Oct2015)
         1   21359755 APPLY           SUCCESS         Database Patch Set Update : 12.1.0.2.5 (21359755)
         1   21555660 ROLLBACK        SUCCESS         Database PSU 12.1.0.2.5, Oracle JavaVM Component (Oct2015)
         1   22674709 APPLY           SUCCESS         Database PSU 12.1.0.2.160419, Oracle JavaVM Component (Apr2016)
         1   22291127 APPLY           SUCCESS         Database Patch Set Update : 12.1.0.2.160419 (22291127)

PBD patches

SQL> select con_id,patch_id,action,status,description from cdb_registry_sqlpatch where con_id = 5 order by action_time;

    CON_ID   PATCH_ID ACTION          STATUS          DESCRIPTION
---------- ---------- --------------- --------------- ----------------------------------------------------------------------------------------------------
         5   21555660 APPLY           SUCCESS         Database PSU 12.1.0.2.5, Oracle JavaVM Component (Oct2015)
         5   21359755 APPLY           SUCCESS         Database Patch Set Update : 12.1.0.2.5 (21359755)
         5   21555660 ROLLBACK        SUCCESS         Database PSU 12.1.0.2.5, Oracle JavaVM Component (Oct2015)
         5   22674709 APPLY           SUCCESS         Database PSU 12.1.0.2.160419, Oracle JavaVM Component (Apr2016)
         5   22291127 APPLY           SUCCESS         Database Patch Set Update : 12.1.0.2.160419 (22291127)

As seen from above outputs both destination CDB and PDB has gone through the same set of patches and are currently at the same patch level.
The source CDB has three PDBs and the PDB called "PDBTHREE" will be moved.

SQL> show pdbs;

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 PDBTHREE                       READ WRITE NO

Close the PBD and unplug it from the source CDB by specifying the metadata xml file.

SQL> alter pluggable database pdbthree close immediate;
Pluggable database altered.

SQL> alter pluggable database pdbthree unplug into '/home/oracle/backup/pdbthree_desc.xml';
Pluggable database altered.

SQL> select pdb_name, status from cdb_pdbs;

PDB_NAME   STATUS
---------- ---------
PDBTHREE   UNPLUGGED
PDB$SEED   NORMAL
ONEPDB     NORMAL
TWOPDB     NORMAL

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 PDBTHREE                       MOUNTED

Drop the PDB with keep file option (default)

SQL> drop pluggable database pdbthree keep datafiles;
Pluggable database dropped.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO

Copy the files over to the destination server. The location of the file is not the same as source location.

[oracle@hpc5 datafile]$ pwd
/opt/app/oracle/oradata/APCDB/3AE6245AE81713AFE0536300A8C02D00/datafile

[oracle@hpc5 datafile]$ scp -C * hpc1:/home/oracle/backup/pdbthree/
oracle@192.168.0.66's password:
o1_mf_pdbthree_cvy3w4f7_.dbf                                                                                                                                100%   10MB  10.0MB/s   00:00
o1_mf_sysaux_cvxvnvnw_.dbf                                                                                                                                  100%  580MB  24.2MB/s   00:24
o1_mf_system_cvxvnvnl_.dbf                                                                                                                                  100%  260MB  18.6MB/s   00:14

On the destination CDB run the compatibility check. In this case the output says that PDB is not compatible.

SQL> SET SERVEROUTPUT ON
DECLARE
SQL>   2  compatible CONSTANT VARCHAR2(3) :=
  3   CASE DBMS_PDB.CHECK_PLUG_COMPATIBILITY(  pdb_descr_file =>'/home/oracle/backup/pdbthree_desc.xml',  pdb_name =>'PDBTHREE')
  4    WHEN TRUE THEN 'YES'
  5    ELSE 'NO'
  6    END;
  7     BEGIN
  8     DBMS_OUTPUT.PUT_LINE(compatible);
  9     END;
 10     /
NO

Checking the plugin violation view shows the following (last two lines has been shortened)

SQL> select name,cause,status,type,action from PDB_PLUG_IN_VIOLATIONS;

NAME       CAUSE           STATUS    TYPE      ACTION
---------- --------------- --------- --------- --------------------------------------------------
PDBTHREE   SQL patch error PENDING   ERROR     Call datapatch to reinstall
PDBTHREE   SQL patch error PENDING   ERROR     Call datapatch to reinstall
PDBTHREE   SQL Patch       PENDING   ERROR     Call datapatch to install in the PDB or the CDB
PDBTHREE   SQL Patch       PENDING   ERROR     Call datapatch to install in the PDB or the CDB

SQL> select status,message from PDB_PLUG_IN_VIOLATIONS;

STATUS    MESSAGE
--------- ------------------------------------------------------------------------------------------------------------------------------------------------------
PENDING    (PSU bundle patch 160419 (Database Patch Set Update : 12.1.0.2.160419 (22291127)): APPLY SUCCESS):  with status  in the PDB.
PENDING    (SQL patch ID/UID 22674709/20057886 (Database PSU 12.1.0.2.160419, Oracle JavaVM Component (Apr2016)): APPLY SUCCESS):  with status  in the PDB.
PENDING   ... Update : 12.1.0.2.160419 (22291127)): Installed in the CDB but not in the PDB.
PENDING   ... Oracle JavaVM Component (Apr2016)): Installed in the CDB but not in the PDB.

Since it has been confirmed that both destination CDB and PDB being plugged in are both patched to same level this violation is ignored.

The destination uses OMF but different paths for PDBs and the root container files. This path is also different from the OMF path at the source CDB. Login to the CDB and set the db_create_file_dest used for PDBs (if a separate OMF path is used for PDBs) and create the PDB with clone and move clause. source_file_name_convert has been used to specify the file mapping where ('/opt/app/oracle/oradata/APCDB/3AE6245AE81713AFE0536300A8C02D00/datafile') refers to PDB data file path at source and ('/home/oracle/backup/pdbthree') refers to the location where data files were copied into.

SQL> alter system set db_create_file_dest='/ssdoracle/oradata/pdbs' scope=memory;
System altered.

SQL> create pluggable database pdbthree as clone using '/home/oracle/backup/pdbthree_desc.xml'
source_file_name_convert=('/opt/app/oracle/oradata/APCDB/3AE6245AE81713AFE0536300A8C02D00/datafile','/home/oracle/backup/pdbthree') move;
Pluggable database created.

SQL> select pdb_name, status from cdb_pdbs;

PDB_NAME        STATUS
--------------- ---------
PDBTHREE        NEW
PDB$SEED        NORMAL
ONEPDB          NORMAL
TWOPDB          NORMAL

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 PDBTHREE                       MOUNTED

SQL> select con_id,name from v$datafile where con_id=5;

    CON_ID NAME
---------- ----------------------------------------------------------------------------------------------------
         5 /ssdoracle/oradata/pdbs/APCDB/3AE70A7E4DC630F7E0534200A8C0F281/datafile/o1_mf_system_cvxzo517_.dbf
         5 /ssdoracle/oradata/pdbs/APCDB/3AE70A7E4DC630F7E0534200A8C0F281/datafile/o1_mf_sysaux_cvxzo529_.dbf
         5 /ssdoracle/oradata/pdbs/APCDB/3AE70A7E4DC630F7E0534200A8C0F281/datafile/o1_mf_pdbthree_cvxzo52b_.dbf

Finally open the PDB

SQL> alter pluggable database pdbthree open;
Pluggable database altered.

SQL>  select pdb_name, status from cdb_pdbs;

PDB_NAME        STATUS
--------------- ---------
PDBTHREE        NORMAL
PDB$SEED        NORMAL
ONEPDB          NORMAL
TWOPDB          NORMAL

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 PDBTHREE                       READ WRITE NO

The CDB doesn't list any more plugin violations

SQL> select name,cause,status,type,action from PDB_PLUG_IN_VIOLATIONS;
no rows selected

Verify the patch levels are reflected

SQL> alter session set container=pdbthree;
Session altered.

SQL> select con_id,patch_id,action,status,description from cdb_registry_sqlpatch where con_id = 5 order by action_time;

    CON_ID   PATCH_ID ACTION     STATUS          DESCRIPTION
---------- ---------- ---------- --------------- ----------------------------------------------------------------------------------------------------
         5   21555660 APPLY      SUCCESS         Database PSU 12.1.0.2.5, Oracle JavaVM Component (Oct2015)
         5   21359755 APPLY      SUCCESS         Database Patch Set Update : 12.1.0.2.5 (21359755)
         5   21555660 ROLLBACK   SUCCESS         Database PSU 12.1.0.2.5, Oracle JavaVM Component (Oct2015)
         5   22674709 APPLY      SUCCESS         Database PSU 12.1.0.2.160419, Oracle JavaVM Component (Apr2016)
         5   22291127 APPLY      SUCCESS         Database Patch Set Update : 12.1.0.2.160419 (22291127)

This concludes the steps for moving a PDB between servers.

Similar Posts
Plugging a SE2 non-CDB as an EE PDB Using File Copying and Remote Link

↧

Plugging a SE2 non-CDB as an EE PDB Using File Copying and Remote Link

September 8, 2016, 12:00 am

≫ Next: Remote Cloning of a PDB

≪ Previous: Move a PDB Between Servers

This post list two methods for plugging a non-CDB on standard edition 2 (SE2) as a pluggable database on enterprise edition (EE) CDB. One is using a XML description file which also includes copying the data files to (if EE database reside in another server). Other method is using a remote link. Reverse of this process, that is EE plugged into SE2 is not possible according to 1631260.1. This is because SE2 has less options than EE. But as long as the options in the SE2 are a subset of EE, moving from SE2 to EE is possible (2020172.1). Current options on the SE2 are

SQL> select comp_id,comp_name,status,version from dba_registry;

COMP_ID    COMP_NAME                                STATUS   VERSION
---------- ---------------------------------------- -------- ----------
CONTEXT    Oracle Text                              VALID    12.1.0.2.0
OWM        Oracle Workspace Manager                 VALID    12.1.0.2.0
XDB        Oracle XML Database                      VALID    12.1.0.2.0
CATALOG    Oracle Database Catalog Views            VALID    12.1.0.2.0
CATPROC    Oracle Database Packages and Types       VALID    12.1.0.2.0

Options on the EE are

SQL> select comp_id,comp_name,status,version from dba_registry;

COMP_ID    COMP_NAME                                STATUS     VERSION
---------- ---------------------------------------- ---------- ---------------
DV         Oracle Database Vault                    VALID      12.1.0.2.0
APEX       Oracle Application Express               VALID      4.2.5.00.08
OLS        Oracle Label Security                    VALID      12.1.0.2.0
SDO        Spatial                                  VALID      12.1.0.2.0
ORDIM      Oracle Multimedia                        VALID      12.1.0.2.0
CONTEXT    Oracle Text                              VALID      12.1.0.2.0
OWM        Oracle Workspace Manager                 VALID      12.1.0.2.0
XDB        Oracle XML Database                      VALID      12.1.0.2.0
CATALOG    Oracle Database Catalog Views            VALID      12.1.0.2.0
CATPROC    Oracle Database Packages and Types       VALID      12.1.0.2.0
JAVAVM     JServer JAVA Virtual Machine             VALID      12.1.0.2.0
XML        Oracle XDK                               VALID      12.1.0.2.0
CATJAVA    Oracle Database Java Packages            VALID      12.1.0.2.0
APS        OLAP Analytic Workspace                  VALID      12.1.0.2.0
XOQ        Oracle OLAP API                          VALID      12.1.0.2.0
RAC        Oracle Real Application Clusters         OPTION OFF 12.1.0.2.0

Both SE2 and EE has been patched to the same level.

Method 1. File Copying
Start the SE2 in read only mode, create the non-CDB descriptor file and shutdown.

SQL> startup mount exclusive;
SQL> alter database open read only;
SQL> exec dbms_pdb.describe(pdb_descr_file=>'/home/oracle/backup/se2_noncdb_desc.xml');
SQL> shutdown imm

Copy the descriptor file the data files to the location where EE CDB resides

scp -C se2_noncdb_desc.xml  192.168.0.99:/home/oracle/backup/

cd /data/oradata/STD12C1/datafile/
scp -C * 192.168.0.99:/home/oracle/backup/

o1_mf_sysaux_byjg3ypy_.dbf                         100%  550MB  25.0MB/s   00:22
o1_mf_system_byjg3q5k_.dbf                         100%  700MB  25.0MB/s   00:28
...

On the EE CDB run a compatibility check against the descriptor file

set serveroutput on
declare
    compa boolean;
    begin
    compa := dbms_pdb.CHECK_PLUG_COMPATIBILITY(pdb_descr_file=>'/home/oracle/backup/se2_noncdb_desc.xml');
 if compa = true then
      dbms_output.put_line('compatible');
 else
    dbms_output.put_line('no compatible');
 end if;
end;
/
no compatible

As seen from the output the compatibility fails. Look in the PDB_PLUG_IN_VIOLATIONS and if the cause for warnings are due to missing options then the incompatibility is ignore-able (2020172.1).

SQL> select name,cause,status,type,action from PDB_PLUG_IN_VIOLATIONS where name='STDPDB' AND STATUS <>'RESOLVED';

NAME CAUSE STATUS TYPE ACTION
-------- ---------- --------- --------- ------------------------------------------------------------
STDPDB OPTION PENDING WARNING Fix the database option in the PDB or the CDB
STDPDB OPTION PENDING WARNING Fix the database option in the PDB or the CDB
STDPDB APEX PENDING WARNING Please contact Oracle Support.

SQL> select status,message from PDB_PLUG_IN_VIOLATIONS where name='STDPDB' AND STATUS <>'RESOLVED';

STATUS MESSAGE
--------- ----------------------------------------------------------------------------------------------------
PENDING Database option APS mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option CATJAVA mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option DV mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option JAVAVM mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option OLS mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option ORDIM mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option SDO mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option XML mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING Database option XOQ mismatch: PDB installed version NULL. CDB installed version 12.1.0.2.0.
PENDING APEX mismatch: PDB installed version NULL CDB installed version 4.2.5.00.08

Plug the SE2 non-CDB using the descriptor file

SQL> create pluggable database se2pdb as clone using '/home/oracle/backup/se2_noncdb_desc.xml'
source_file_name_convert=('/data/oradata/STD12C1/datafile','/home/oracle/backup') move;

Pluggable database created.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
 5 SE2PDB                         MOUNTED

Chane to the PDB container and run noncdb_to_pdb.sql script

SQL> ALTER SESSION SET CONTAINER=se2pdb;
SQL> show con_name;

CON_NAME
------------------------------
SE2PDB

SQL> @?/rdbms/admin/noncdb_to_pdb.sql

When the script completes open the PDB

SQL> alter pluggable database open;

SQL> show pdbs;

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
5 SE2PDB                         READ WRITE NO

PDB opens without any restriction and is ready to use. EE CDB was setup using OMF, as such the plugged SE2 DB's datafile and tempfile will be created in a OMF structure with a new GUID.

SQL> select name from v$tempfile;

NAME
----------------------------------------------------------------------------------------------------
/opt/app/oracle/oradata/CGCDB/3B61A02E318577C4E0536300A8C051DC/datafile/o1_mf_temp_cwg1lqd0_.dbf

1 row selected.

SQL> select name from v$datafile;

NAME
----------------------------------------------------------------------------------------------------
/opt/app/oracle/oradata/CGCDB/datafile/o1_mf_undotbs1_cvchzywd_.dbf
/opt/app/oracle/oradata/CGCDB/3B61A02E318577C4E0536300A8C051DC/datafile/o1_mf_system_cwg1lqcv_.dbf
/opt/app/oracle/oradata/CGCDB/3B61A02E318577C4E0536300A8C051DC/datafile/o1_mf_sysaux_cwg1lqcz_.dbf
/opt/app/oracle/oradata/CGCDB/3B61A02E318577C4E0536300A8C051DC/datafile/o1_mf_users_cwg1lqd0_.dbf
/opt/app/oracle/oradata/CGCDB/3B61A02E318577C4E0536300A8C051DC/datafile/o1_mf_tools_cwg1lqd1_.dbf

The files associated with the undo and temp tablespaces on the non-CDB will not be used in the plugged in PDB.

Method 2. Remote Link
In this method a remote link created from the EE CDB to SE2 non-CDB and used to create the PDB.

stdtns =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.0.66)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = std12c1)
    )
  )

SQL> create database link std_link connect to system identified by system using 'STDTNS';
Database link created.

Validate the link by running a query against the remote SE2 DB

SQL> select instance_name from v$instance@std_link;

INSTANCE_NAME
----------------
std12c1

As OMF is used no other steps are needed to create the PDB. Run the create PDB statement

SQL>  create pluggable database stdpdb from std12c1@std_link;
Pluggable database created.

SQL> show pdbs;

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 STDPDB                         MOUNTED

Change to the newly created PDB, run the noncdb_to_pdb script and finally open the PDB

SQL> ALTER SESSION SET CONTAINER=stdpdb;

SQL> show con_name;

CON_NAME
------------------------------
STDPDB

SQL> @?/rdbms/admin/noncdb_to_pdb.sql

SQL> alter pluggable database open;

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 STDPDB                         READ WRITE NO

Similar to method 1 and plugging violation due to missing options are ignore-able.
Unlike the method 1, in method 2 the undo tablespace within the PDB will be offline.

SQL> select tablespace_name,status from dba_tablespaces order by 2,1;

TABLESPACE_NAME                STATUS
------------------------------ ---------
UNDOTBS1                       OFFLINE

This is due to the fact that both non-CDB and CDB where it's plugged having the same name for undo tablespace and undo tablespaces are not local to PDB but common to entire CDB. Unfortunately it's not possible to drop this within the PDB (2067414.1). However querying data files within the PDB shows the CDB's undo tablespace's data file

SQL> select name from v$datafile;

NAME
----------------------------------------------------------------------------------------------------
/opt/app/oracle/oradata/CGCDB/datafile/o1_mf_undotbs1_cvchzywd_.dbf
/opt/app/oracle/oradata/CGCDB/3BEEE6493046715CE0536300A8C0B4F0/datafile/o1_mf_system_cx0l5krb_.dbf
/opt/app/oracle/oradata/CGCDB/3BEEE6493046715CE0536300A8C0B4F0/datafile/o1_mf_sysaux_cx0l5krf_.dbf

Useful Metlink Notes
"OPTION WARNING Database option mismatch: PDB installed version NULL" in PDB_PLUG_IN_VIOLATIONS [ID 2020172.1]
Unplug an Enterprise Edition PDB and Plug into a Standard Edition CDB says not compatible when using DBMS_PDB.CHECK_PLUG_COMPATIBILITY [ID 1631260.1]

Related Post
Move a PDB Between Servers

↧

Remote Cloning of a PDB

September 18, 2016, 12:00 am

≫ Next: Insert a File as BLOB Using DBMS_LOB

≪ Previous: Plugging a SE2 non-CDB as an EE PDB Using File Copying and Remote Link

Similar to non-CDB, PDB too could cloned over a remote link. In this case both source and remote DBs are CDBs and one PDB is cloned on the local DB. As the first step create a TNS entry and a link on the local DB. The remote PDB is called PDB1K

PDB1KTNS =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.0.88)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = pdb1k)
    )
  )

SQL> create database link pdb1k_link connect to  system identified by system using 'PDB1KTNS';

Validate the link by querying a view on the remote PDB


SQL> select name from v$pdbs@pdb1k_link;

NAME
------------------------------
PDB1K

If OMF is used nothing else is needed and PDB could be cloned. However in this case a data files of the remotely cloned PDBs are stored separately. To achieve that set the db_create_dest parameter to desired location with scope set to memory.

SQL> alter system set db_create_file_dest='/opt/app/oracle/oradata/remoteclones' scope=memory;

SQL> show parameter db_create

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest                  string      /opt/app/oracle/oradata/remoteclones

Create the PDB, the new PDB is named PDB1KRMT.

SQL> create pluggable database pdb1krmt from pdb1k@pdb1k_link;

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 PDB1KRMT                       MOUNTED

Finally open the PDB

SQL> alter pluggable database pdb1krmt open;

SQL> show pdbs;

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 ONEPDB                         READ WRITE NO
         4 TWOPDB                         READ WRITE NO
         5 PDB1KRMT                       READ WRITE NO

Verify the PDB data files are created in the intended location

SQL>  select name from v$datafile;

NAME
------------------------------------------------------------------------------------------------------------------------
/opt/app/oracle/oradata/CGCDB/datafile/o1_mf_undotbs1_cvchzywd_.dbf
/opt/app/oracle/oradata/remoteclones/CGCDB/3B5F12ED0B6F4762E0536300A8C0A85F/datafile/o1_mf_system_cwfq6dd5_.dbf
/opt/app/oracle/oradata/remoteclones/CGCDB/3B5F12ED0B6F4762E0536300A8C0A85F/datafile/o1_mf_sysaux_cwfq6ddt_.dbf
/opt/app/oracle/oradata/remoteclones/CGCDB/3B5F12ED0B6F4762E0536300A8C0A85F/datafile/o1_mf_pdb1ktbs_cwfq6ddv_.dbf

Using the USER_TABLESPACES clause available on 12.1.0.2 it is possible to clone the new PDB only with a subset of tablespaces. Assume that original PDB has 3 application specific tablespaces.

SQL> select tablespace_name,status from dba_tablespaces;

TABLESPACE_NAME                STATUS
------------------------------ ---------
SYSTEM                         ONLINE
SYSAUX                         ONLINE
TEMP                           ONLINE
APP1                           ONLINE
APP2                           ONLINE
APP3                           ONLINE

Only 2 of them are wanted in the newly cloned PDB. It is possible to include just these two tablespaces in the user_tablespaces clause excluding all other tablespaces.

create pluggable database pdb1krmt from pdb1k@pdb1k_link USER_TABLESPACES=('APP1','APP3');

Tablespace name exists but status will be offline with data file missing as well.

SQL> select tablespace_name,status from dba_tablespaces;

TABLESPACE_NAME                STATUS
------------------------------ ---------
SYSTEM                         ONLINE
SYSAUX                         ONLINE
TEMP                           ONLINE
APP1                           ONLINE
APP2                           OFFLINE
APP3                           ONLINE

SQL> select tablespace_name,status,file_name from dba_data_files;

TABLESPACE_NAME                STATUS    FILE_NAME
------------------------------ --------- ----------------------------------------------------------------------------------------------------
SYSTEM                         AVAILABLE /opt/app/oracle/oradata/CGCDB/3BED1E0F3E63672CE0536300A8C0356F/datafile/o1_mf_system_cx0bpkwo_.dbf
SYSAUX                         AVAILABLE /opt/app/oracle/oradata/CGCDB/3BED1E0F3E63672CE0536300A8C0356F/datafile/o1_mf_sysaux_cx0bpkwy_.dbf
APP1                           AVAILABLE /opt/app/oracle/oradata/CGCDB/3BED1E0F3E63672CE0536300A8C0356F/datafile/o1_mf_app1_cx0bpkwz_.dbf
APP3                           AVAILABLE /opt/app/oracle/oradata/CGCDB/3BED1E0F3E63672CE0536300A8C0356F/datafile/o1_mf_app3_cx0bpkx1_.dbf
APP2                           AVAILABLE /opt/app/oracle/product/12.1.0/dbhome_2/dbs/MISSING00122

Offline tablespace could be dropped to clean up the new PDB

SQL> drop tablespace app2 including contents and datafiles cascade constraints;

Same could be done when plugging non-CDB as PDBs as well.

SQL> create pluggable database stdpdb from std12c1@std_link USER_TABLESPACES=('APP1','APP3');

Run the post cloning steps and verify the tablespace list

SQL> select tablespace_name,status from dba_tablespaces order by 2,1;

TABLESPACE_NAME                STATUS
------------------------------ ---------
APP2                           OFFLINE
TOOLS                          OFFLINE
UNDOTBS1                       OFFLINE
USERS                          OFFLINE
APP1                           ONLINE
APP3                           ONLINE
SYSAUX                         ONLINE
SYSTEM                         ONLINE
TEMP                           ONLINE

SQL> select tablespace_name,status,file_name from dba_data_files;

TABLESPACE_NAME                STATUS    FILE_NAME
------------------------------ --------- ----------------------------------------------------------------------------------------------------
SYSTEM                         AVAILABLE /opt/app/oracle/oradata/CGCDB/3BEC00E8AA7862ECE0536300A8C0B5F7/datafile/o1_mf_system_cx060v17_.dbf
SYSAUX                         AVAILABLE /opt/app/oracle/oradata/CGCDB/3BEC00E8AA7862ECE0536300A8C0B5F7/datafile/o1_mf_sysaux_cx060v18_.dbf
USERS                          AVAILABLE /opt/app/oracle/product/12.1.0/dbhome_2/dbs/MISSING00114
TOOLS                          AVAILABLE /opt/app/oracle/product/12.1.0/dbhome_2/dbs/MISSING00115
APP1                           AVAILABLE /opt/app/oracle/oradata/CGCDB/3BEC00E8AA7862ECE0536300A8C0B5F7/datafile/o1_mf_app1_cx060v1b_.dbf
APP2                           AVAILABLE /opt/app/oracle/product/12.1.0/dbhome_2/dbs/MISSING00117
APP3                           AVAILABLE /opt/app/oracle/oradata/CGCDB/3BEC00E8AA7862ECE0536300A8C0B5F7/datafile/o1_mf_app3_cx060v1c_.dbf

Undo tablespace within the PDB cannot be removed (2067414.1).

SQL> drop tablespace UNDOTBS1 including contents and datafiles cascade constraints;
drop tablespace UNDOTBS1 including contents and datafiles cascade constraints
*
ERROR at line 1:
ORA-30013: undo tablespace 'UNDOTBS1' is currently in use

Offline undo tablespace in this case is the undo tablespace on non-CDB. This is because the CDB undo tablespace name and cloned non-CDB tablespace name is the same and undo is not local to PDB but common to entire CDB. It was not possible to get rid of the undotbs1 offline status even after switching the default undo tablespace of the CDB to a different undo tablespace.
This clause does not apply to the SYSTEM, SYSAUX, or TEMP tablespaces.

Similar Posts
Move a PDB Between Servers
Plugging a SE2 non-CDB as an EE PDB Using File Copying and Remote Link

↧

Insert a File as BLOB Using DBMS_LOB

September 28, 2016, 12:00 am

≫ Next: Moving PDB from RAC CDB to Single Instance CDB

≪ Previous: Remote Cloning of a PDB

Following script could be used to insert a file as a BLOB into a table. The file to be inserted must reside in the location referred by the database directory LOADF. File name in this case data.log. File is inserted to table called lobins which only has single column which is of BLOB type.

declare

  file_name varchar2(100) := 'data.log';
  db_dir_name varchar2(100) := 'LOADF';

  dest_loc  BLOB := empty_blob();
  src_loc   BFILE := BFILENAME(db_dir_name, file_name);
  destoff number := 1;
  srcoff number :=1;

begin

  DBMS_LOB.OPEN(src_loc, DBMS_LOB.FILE_READONLY);
  DBMS_LOB.CREATETEMPORARY(lob_loc => dest_loc, cache => false);
  DBMS_LOB.OPEN(dest_loc, DBMS_LOB.LOB_READWRITE);
  DBMS_LOB.LOADBLOBFROMFILE (dest_loc,src_loc,DBMS_LOB.getLength(src_loc),destoff,srcoff);

  insert into lobins values (dest_loc);
  commit;

  DBMS_LOB.CLOSE(dest_loc);
  DBMS_LOB.CLOSE(src_loc);

  end;
  /

↧

Moving PDB from RAC CDB to Single Instance CDB

October 1, 2016, 12:00 am

≫ Next: Change in 12c JDBC Behavior - setDate & getDate Does Not Truncate Timestamp

≪ Previous: Insert a File as BLOB Using DBMS_LOB

The RAC CDB and the non-RAC CDB are both 12.1.0.2 SE2 instances. Currently the single instance CDB doesn't have any PDB plugged on it. SE2 CDBs only support single tenancy due to license restriction. The RAC CDB has a PDB available on both instances.

INSTANCE     NAME         OPEN_MODE  STATUS    RES
------------ ------------ ---------- --------- ---
stdcdb1      PDB$SEED     READ ONLY  NORMAL    NO
stdcdb1      STDPDB       READ WRITE NORMAL    NO
stdcdb2      PDB$SEED     READ ONLY  NORMAL    NO
stdcdb2      STDPDB       READ WRITE NORMAL    NO

Both CDBs has same components except for RAC components which is set not available on the single instance CDB. DB registry of the RAC CDB

SQL> select comp_name,status from dba_registry order by 1;

COMP_NAME                                STATUS
---------------------------------------- -----------
JServer JAVA Virtual Machine             VALID
Oracle Application Express               VALID
Oracle Database Catalog Views            VALID
Oracle Database Java Packages            VALID
Oracle Database Packages and Types       VALID
Oracle Multimedia                        VALID
Oracle Real Application Clusters         VALID
Oracle Text                              VALID
Oracle Workspace Manager                 VALID
Oracle XDK                               VALID
Oracle XML Database                      VALID

DB Registry of the non-RAC CDB

SQL> select comp_name,status from dba_registry order by 1;

COMP_NAME                                STATUS
---------------------------------------- -----------
JServer JAVA Virtual Machine             VALID
Oracle Application Express               VALID
Oracle Database Catalog Views            VALID
Oracle Database Java Packages            VALID
Oracle Database Packages and Types       VALID
Oracle Multimedia                        VALID
Oracle Real Application Clusters         OPTION OFF
Oracle Text                              VALID
Oracle Workspace Manager                 VALID
Oracle XDK                               VALID
Oracle XML Database                      VALID

Process of moving the PDB is similar to earlier post except for few RAC and ASM related tasks. To summarize, before the PDB could be moved from currently plugged in CDB, it must be stopped, unplugged and dropped. The unplugging will create a PDB descriptor file and on in the DB file names will appear in lower case while rest of the path appear upper case.

cat stdpdb_desc.xml | grep path
+DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/DATAFILE/system.291.894734477
+DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/DATAFILE/sysaux.292.894734481
+DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/TEMPFILE/temp.295.922121143
+DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/DATAFILE/users.293.894734495
+DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/DATAFILE/tapp.294.894734495

However when data files are copied out of ASM using asmcmd they will have the names in upper case (e.g. SYSTEM.291.894734477). As a result plugging will fail as it is unable to identify the file name

create pluggable database stdpdb as clone using '/home/oracle/stdcdb/datafiles/stdpdb_desc.xml'
*
ERROR at line 1:
ORA-19505: failed to identify file "/home/oracle/stdcdb/datafiles/system.291.894734477"
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory

SQL> ! ls "/home/oracle/stdcdb/datafiles/system.291.894734477"
ls: /home/oracle/stdcdb/datafiles/system.291.894734477: No such file or directory

SQL> ! ls -l /home/oracle/stdcdb/datafiles/*
-rw-r--r-- 1 oracle oinstall      6925 Sep 12 10:47 /home/oracle/stdcdb/datafiles/stdpdb_desc.xml
-rw-r----- 1 oracle oinstall 550510592 Sep 12 10:44 /home/oracle/stdcdb/datafiles/SYSAUX.292.894734481
-rw-r----- 1 oracle oinstall 293609472 Sep 12 10:45 /home/oracle/stdcdb/datafiles/SYSTEM.291.894734477
-rw-r----- 1 oracle oinstall  20979712 Sep 12 10:45 /home/oracle/stdcdb/datafiles/TAPP.294.894734495
-rw-r----- 1 oracle oinstall   5251072 Sep 12 10:45 /home/oracle/stdcdb/datafiles/USERS.293.894734495

So before the plugging rename the file names same as on descriptor file.

 mv SYSTEM.291.894734477 system.291.894734477
 mv SYSAUX.292.894734481 sysaux.292.894734481
 mv TAPP.294.894734495 tapp.294.894734495
 mv USERS.293.894734495 users.293.894734495

Also in ASM datafiles and tempfiles have separate paths, as a result the tempfile location must also specified in the source_file_name convert even though file itself is not necessary to be copied to new location. Temp file get dropped when the PDB is dropped even though keep data file options is used.

drop pluggable database stdpdb keep datafiles
Deleted Oracle managed file +DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/TEMPFILE/temp.295.922364799
Completed: drop pluggable database stdpdb keep datafiles

So the final command to plug the PDB into single instance will have two file name convert entires, one for data files and another for temp files

create pluggable database stdpdb as clone using '/home/oracle/stdcdb/datafiles/stdpdb_desc.xml'
source_file_name_convert=(
'+DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/DATAFILE','/home/oracle/stdcdb/datafiles',
'+DATA/STDCDB/238FA674AFE26E9DE0535E00A8C0F5A1/TEMPFILE','/home/oracle/stdcdb/datafiles') move;

Once the PDB is plugged open it

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 STDPDB                         MOUNTED

SQL> alter pluggable database stdpdb open;

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 STDPDB                         READ WRITE NO

SQL> select name from v$datafile;

NAME
-----------------------------------------------------------------------------------------------
/ssdoracle/oradata/STDCDB/datafile/o1_mf_undotbs1_cx2j8d05_.dbf
/ssdoracle/oradata/STDCDB/3C4F4864ACAE0C3DE0534200A8C05C7A/datafile/o1_mf_system_cxf6fp5g_.dbf
/ssdoracle/oradata/STDCDB/3C4F4864ACAE0C3DE0534200A8C05C7A/datafile/o1_mf_sysaux_cxf6fp5h_.dbf
/ssdoracle/oradata/STDCDB/3C4F4864ACAE0C3DE0534200A8C05C7A/datafile/o1_mf_users_cxf6fp5j_.dbf
/ssdoracle/oradata/STDCDB/3C4F4864ACAE0C3DE0534200A8C05C7A/datafile/o1_mf_tapp_cxf6fp5k_.dbf

Looking at the PDB violation view will show the warning related to missing RAC option.

SQL> select name,status,action,message from PDB_PLUG_IN_VIOLATIONS where cause='OPTION';

NAME     STATUS    ACTION                                             MESSAGE
-------- --------- -------------------------------------------------- -------------------------------------------------------------------------------------------
STDPDB   PENDING   Fix the database option in the PDB or the CDB      Database option RAC mismatch: PDB installed version 12.1.0.2.0. CDB installed version NULL.

SQL> alter session set container=stdpdb;

SQL> show con_name

CON_NAME
---------
STDPDB

SQL> select comp_name,status from dba_registry order by 1;

COMP_NAME                                STATUS
---------------------------------------- -----------
JServer JAVA Virtual Machine             VALID
Oracle Application Express               VALID
Oracle Database Catalog Views            VALID
Oracle Database Java Packages            VALID
Oracle Database Packages and Types       VALID
Oracle Multimedia                        VALID
Oracle Real Application Clusters         VALID
Oracle Text                              VALID
Oracle Workspace Manager                 VALID
Oracle XDK                               VALID
Oracle XML Database                      VALID

11 rows selected.

SQL> exec dbms_registry.OPTION_OFF('RAC');

SQL> select comp_name,status from dba_registry order by 1;

COMP_NAME                                STATUS
---------------------------------------- -----------
JServer JAVA Virtual Machine             VALID
Oracle Application Express               VALID
Oracle Database Catalog Views            VALID
Oracle Database Java Packages            VALID
Oracle Database Packages and Types       VALID
Oracle Multimedia                        VALID
Oracle Real Application Clusters         OPTION OFF
Oracle Text                              VALID
Oracle Workspace Manager                 VALID
Oracle XDK                               VALID
Oracle XML Database                      VALID

Close and open the PDB. The PDB violation view will show the issue has been resolved.

SQL> select name,status,action,message from PDB_PLUG_IN_VIOLATIONS where cause='OPTION';

NAME     STATUS    ACTION                                             MESSAGE
-------- --------- -------------------------------------------------- -------------------------------------------------------------------------------------------
STDPDB   RESOLVED  Fix the database option in the PDB or the CDB      Database option RAC mismatch: PDB installed version 12.1.0.2.0. CDB installed version NULL.

This concludes the moving of PDB from RAC CDB to single instance CDB.

Related Posts
Move a PDB Between Servers
Plugging a SE2 non-CDB as an EE PDB Using File Copying and Remote Link

↧

Change in 12c JDBC Behavior - setDate & getDate Does Not Truncate Timestamp

October 15, 2016, 12:00 am

≫ Next: Plugging a Non-CDB (pre-12c) into CDB Using Transportable DB (TDB) / Tablespaces (TTS)

≪ Previous: Moving PDB from RAC CDB to Single Instance CDB

The 12c driver (12.1.0.2, ojdbc7.jar) certified for JDK7 and JDK8 does not truncate or set to "00:00:00" the time component when called with setDate and getDate methods. Same could be observed for 12c driver for JDK 6 as well (ojdbc6.jar). The test code given at the end of the post could be used to demonstrate this change in behavior compared to 11g2 (11.2.0.4, ojdbc6.jar certified for JDK6, JDK7 and JDK8) driver. Various driver/database/JDK compatible matrix could be found here.
The test java code runs on JDK 8 and database is a 12.1.0.2 CDB. The only thing changes between tests is the JDBC driver.
First the test is run with 11.2 (11.2.0.4) driver. The output (truncated here) will have no time component on the values inserted or returned by setDate and getDate methods. Timestamp method is used to check what is stored in the table. The output is of the following form: the method (getDate or getTimestamp) used, the driver used to insert the value and value returned by get method used.

Date 11.2.0.4.0 2016-10-12 00:00:00
Timestamp 11.2.0.4.0 2016-10-12 00:00:00.0

There's no issues to reading these values with a 12.1 (12.1.0.2, ojdbc7.jar) driver, the output will be the same (run the test commenting calls to delete and insert methods)

Date 11.2.0.4.0 2016-10-12 00:00:00
Timestamp 11.2.0.4.0 2016-10-12 00:00:00.0

Changing the driver to 12c results in date being inserted with a time component (run test by un-commenting the previously commented methods).

Date 12.1.0.2.0 2016-10-12 12:26:12
Timestamp 12.1.0.2.0 2016-10-12 12:26:12.0

Depending on the application logic upgrade to 12c driver could cause issues due to this change in behavior. In such cases going back to 11.2 driver may work in some cases, as getDate would truncate the time component. But the actual value stored has the time component, shown here in the getTimestamp method, so the behavior is not entirely reversible by reverting to the 11.2 driver.

Date 12.1.0.2.0 2016-10-12 00:00:00
Timestamp 12.1.0.2.0 2016-10-12 12:26:12.0

There's few bug reports on MOS related this issue (Bug 19297927, Bug 20551186, 1944845.1). But the final outcome seem to be that this deliberate (Bug 17766200) and expected behavior in 12c driver (2177909.1) and that oracle documentation isn't reflecting it (Bug 18124680).

Therefore, if the use of 12c driver is a must then oracle has few workarounds and patches for this. One option is to use a calendar object and set the time components to 0 (1944845.1). Other is to apply patch 21161279 (2177909.1). The patch 21161279 supersedes patch 19297927 which only patched the setDate method. So getDate will return values inserted with time component. Once the patch 21161279 is applied following JVM option must be set in order for the set/get Date method to behave as 11.2 driver's methods. Patch has no effect unless this option is set

-Doracle.jdbc.DateZeroTime=true

Once patched and JVM option is set the run the test with 12.1 driver. The time components will be set to 0.

Date 12.1.0.2.0 2016-10-12 00:00:00
Timestamp 12.1.0.2.0 2016-10-12 00:00:00.0

This behavior and output values are same as 11.2 driver.
Same patch and JVM option could be used for 12.1 driver for JDK 6 as well (ojdbc6.jar).

Useful metalink notes
getDate() Gets Time After Upgrading To 12c [ID 2177909.1]
JDBC 12c Adds Timestamp to java.sql.Date After Upgrading From 11.2.0.4 [ID 1944845.1]
Bug 19297927 : CHANGE OF BEHAVIOR IN JDBC 12.1 DUE TO BUG 14389749 CAUSES QUERIES TO FAIL
Bug 17766200 : GETDATE AND SETDATE DO NOT TRUNCATE TIME IN JDBC 12C
Bug 20551186 : GETDATE DIFFERS WITH THE JDBC DRIVER VERSION 11G AND 12C
Bug 18124680 : GETDATE AND SETDATE DO NOT TRUNCATE TIME IN JDBC 12C BUT DOC STATES OTHERWISE
Bug 17228297 : JDBC DRIVER OJDBC*.JAR 12.1.0.1.0 SPECIFICATION VIOLATION REGRESSION

Related Post
java.sql.SQLException: Could not commit with auto-commit set on When Using 12c JDBC Driver

Test Table DDL

create table datetest (version varchar2(20), indate date);

Java Test Code

public class OJDBC7DateTest {

    public static void main(String[] args) throws SQLException {

        OracleDataSource ds = new OracleDataSource();
        ds.setUser("asanga");
        ds.setPassword("asa");
        ds.setURL("jdbc:oracle:thin:@192.168.0.66:1521/onepdb");

        Connection con = ds.getConnection();
        con.setAutoCommit(false);

        System.out.println("Auto commit status : " + con.getAutoCommit());
        DatabaseMetaData meta = con.getMetaData();

        System.out.println("Driver Name " + meta.getDriverName());
        System.out.println("Driver Version " + meta.getDriverVersion());
        System.out.println("Driver Major Version " + meta.getDriverMajorVersion());
        System.out.println("Driver Minor Version " + meta.getDriverMinorVersion());
        System.out.println("Database Major Version " + meta.getDatabaseMajorVersion());
        System.out.println("Database Minor Version " + meta.getDatabaseMinorVersion());

        System.out.println("delete");
        delete(con);

        System.out.println("insert");
        insert(con,meta.getDriverVersion());

        System.out.println("select");
        select(con);

        con.close();
    }

    public static void select(Connection con) throws SQLException {

        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

        PreparedStatement ps = con.prepareStatement("select * from datetest order by 1");
        ResultSet rs = ps.executeQuery();
        while (rs.next()) {

            System.out.println("Date " + rs.getString(1) + "" + sdf.format(rs.getDate(2)));
            System.out.println("Timestamp " + rs.getString(1) + "" + rs.getTimestamp(2));
        }
        rs.close();
        ps.close();

    }

    public static void insert(Connection con, String version) throws SQLException {

        PreparedStatement ps = con.prepareStatement("insert into datetest values(?, ?)");
        Date cd = new Date(System.currentTimeMillis());
        ps.setString(1, version);;
        ps.setDate(2, cd);
        ps.execute();
        con.commit();
        ps.close();

    }

    public static void delete(Connection con) throws SQLException {

        PreparedStatement ps = con.prepareStatement("delete from datetest");
        ps.execute();
        con.commit();
        ps.close();

    }
}

↧

Plugging a Non-CDB (pre-12c) into CDB Using Transportable DB (TDB) / Tablespaces (TTS)

November 1, 2016, 12:00 am

≫ Next: ORA-39127: unexpected error from call to "SYS"."DBMS_JVM_EXP_PERMS"."GRANT_SYSPRIVS_EXP"

≪ Previous: Change in 12c JDBC Behavior - setDate & getDate Does Not Truncate Timestamp

In previous post it was shown how a 12c non-CDB could be plugged in as a PDB. This post shows steps for plugging a pre-12c DB (anything pre-12c is by default non-CDB) as a PBD using transportable DB (TDB) and tablespace (TTS) methods. Both methods have similar steps, only difference being in TBD entire DB is transported while in TTS method only a set of tablespaces plugged in. In this case enterprise edition RAC DB (called ent114) of 11.2.0.4 version will be plugged into a single instance CDB using TDB/TTS methods.

Both methods requires creating a PDB before the transporting could begin. The tablespaces are attached to this "pre-existing" PDB. In this case it is called PDB114.

SQL> create pluggable database pdb114 admin user admin identified by admin;
SQL> alter pluggable database pdb114 open;

Following information and steps are common for both methods. The 11.2 database has following tablespaces. The user defined tablespaces are in bold.

SQL> select TABLESPACE_NAME,BLOCK_SIZE from dba_tablespaces order by 1;

TABLESPACE_NAME                BLOCK_SIZE
------------------------------ ----------
EXAMPLE                              8192
NEWTBS                               8192
SQLT                                 8192
SYSAUX                               8192
SYSTEM                               8192
TBS2K                                2048
TBS32K                              32768
TEMP                                 8192
TEST                                 8192
UNDOTBS1                             8192
UNDOTBS2                             8192
USERS                                8192

There are two tablespaces with non-default block sizes. Before the transport create non-default memory pools in the CDB as well.
Secondly also check if the endieness of the source and target systems are the same. If they are different, conversions is required before transported data files could be attached.

SQL> SELECT d.PLATFORM_NAME, ENDIAN_FORMAT FROM V$TRANSPORTABLE_PLATFORM tp, V$DATABASE d WHERE tp.PLATFORM_NAME = d.PLATFORM_NAME;  

PLATFORM_NAME        ENDIAN_FORMAT
-------------------- --------------
Linux x86 64-bit     Little

This is where the steps common for both TDB and TTS ends. Next is the steps specific to TDB method.

Plugging pre-12c DB using TDB
Put the user defined tablespaces to read only mode.

alter tablespace EXAMPLE read only;
alter tablespace NEWTBS read only;
alter tablespace SQLT read only;
alter tablespace TBS2K read only;
alter tablespace TBS32K read only;
alter tablespace TEST read only;
alter tablespace USERS read only;

SQL> select tablespace_name,status from dba_tablespaces;

TABLESPACE_NAME                STATUS
------------------------------ ---------
SYSTEM                         ONLINE
SYSAUX                         ONLINE
UNDOTBS1                       ONLINE
TEMP                           ONLINE
UNDOTBS2                       ONLINE
USERS                          READ ONLY
EXAMPLE                        READ ONLY
TEST                           READ ONLY
SQLT                           READ ONLY
NEWTBS                         READ ONLY
TBS2K                          READ ONLY
TBS32K                         READ ONLY

While the user defined tablespaces are in read only mode run a full export of the DB with transportable = always and version = 12 options. This option is supported on standard edition databases. Following expdp command is used in this case and tail end of the export log output is shown as well.

expdp system full=y dumpfile=ent114.dmp directory=EXEC_DIR transportable=always version=12 logfile=ent114.log

******************************************************************************
Dump file set for SYSTEM.SYS_EXPORT_FULL_01 is:
  /usr/local/exdata/ent114.dmp
******************************************************************************
Datafiles required for transportable tablespace EXAMPLE:
  +DATA/ent11g2/datafile/example.268.785857285
Datafiles required for transportable tablespace NEWTBS:
  +DATA/ent11g2/datafile/newtbs.273.888232727
Datafiles required for transportable tablespace SQLT:
  +DATA/ent11g2/datafile/sqlt.269.826463789
Datafiles required for transportable tablespace TBS2K:
  +DATA/ent11g2/datafile/tbs2k.272.921767965
Datafiles required for transportable tablespace TBS32K:
  +DATA/ent11g2/datafile/tbs32k.271.921768047
Datafiles required for transportable tablespace TEST:
  +DATA/ent11g2/datafile/test.274.888250337
Datafiles required for transportable tablespace USERS:
  +DATA/ent11g2/datafile/users.264.785694801
Job "SYSTEM"."SYS_EXPORT_FULL_01" completed with 1 error(s) at Mon Sep 5 14:52:08 2016 elapsed 0 00:05:34

The data files listed for transport must be copied out of the ASM disk group. There are many ways to do this, easiest could be to use cp with asmcmd. One example is shown here

ASMCMD> cp TBS2K.272.921767965 /home/grid/backup/
copying +DATA/ent11g2/datafile/TBS2K.272.921767965 -> /home/grid/backup/TBS2K.272.921767965

Once all the data files required for transport is copied out of ASM, the tablespaces made read only earlier could be made read write again.

alter tablespace EXAMPLE read write;
alter tablespace NEWTBS read write;
alter tablespace SQLT read write;
alter tablespace TBS2K read write;
alter tablespace TBS32K read write;
alter tablespace TEST read write;
alter tablespace USERS read write;

Next step is moving of data files copied out of ASM eariler to the actual location where PDB reside. In this case the PDB reside on a remote server and data file location for the PDB is

/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile

Therefore the data files are copied over the same location.

scp * oracle@192.168.0.99:/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/

Create a database directory in the PDB and copy the dump file created by the expdp to location refereed by the DB directory. This will be used during the import. Since there's no conversion needed due to endian format being the same the import can go ahead without any additional work. The parameter file used for import is shown below. The TRANSPORT_DATAFILES has comma separated list of datafiles with their new paths.

cat import.par
FULL=Y
DUMPFILE=ent114.dmp
DIRECTORY=ora_dump
TRANSPORT_DATAFILES=
'/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/EXAMPLE.268.785857285',
'/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/NEWTBS.273.888232727',
'/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/SQLT.269.826463789',
'/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/TBS2K.272.921767965',
'/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/TBS32K.271.921768047',
'/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/TEST.274.888250337',
'/opt/app/oracle/oradata/CXCDB/3BC31BB9D1A967C7E0536300A8C0D124/datafile/USERS.264.785694801'
LOGFILE=ent114imp.log

The DB is imported using the parameter file. The objects that already exists could result in warnings however the import completes successfully.

impdp system@pdb114 parfile=import.par

At the end of the import the user defined tablespaces are plugged in and online.

SQL> SELECT tablespace_name, plugged_in, status FROM   dba_tablespaces;

TABLESPACE_NAME                PLU STATUS
------------------------------ --- ---------
SYSTEM                         NO  ONLINE
SYSAUX                         NO  ONLINE
TEMP                           NO  ONLINE
EXAMPLE                        YES ONLINE
NEWTBS                         YES ONLINE
SQLT                           YES ONLINE
TBS2K                          YES ONLINE
TBS32K                         YES ONLINE
TEST                           YES ONLINE
USERS                          YES ONLINE

All users in the pre-12c DB are created as local users of the PDB. In the default database properties are not affected, for example in the pre-12c DB the default DB tablespace was users but after the transport the PDB still retain it's original default tablespace.

SQL> SELECT PROPERTY_NAME,PROPERTY_VALUE FROM database_properties where property_name like 'DEFAULT%';

PROPERTY_NAME                  PROPERTY_VALUE
------------------------------ ----------------
DEFAULT_TEMP_TABLESPACE        TEMP
DEFAULT_PERMANENT_TABLESPACE   SYSTEM

At this stage the PDB is ready for use with pre-12c DB plugged in.

Plugging pre-12c DB using TTS
This method uses transportable tablespaces to copy the user defined tablespaces to the PDB location and plug them to the PDB created. Same user defined tablespaces mentioned in earlier methods are used in this case as well. Before tablespaces could be transported verify they are self contained. DBMS_TTS.TRANSPORT_SET_CHECK could be used for this, however unlike in previous post strict or full containment check is also performed. Oracle admin guide says "for strict or full containment, you must additionally set the TTS_FULL_CHECK parameter to TRUE. The strict or full containment check is for cases that require capturing not only references going outside the transportable set, but also those coming into the set. Tablespace Point-in-Time Recovery (TSPITR) is one such case where dependent objects must be fully contained or fully outside the transportable set".

SQL> exec DBMS_TTS.TRANSPORT_SET_CHECK('EXAMPLE,NEWTBS,SQLT,TBS2K,TBS32K,TEST,USERS', TRUE,TRUE);

Check for any violations

SQL> SELECT * FROM TRANSPORT_SET_VIOLATIONS;
no rows selected

If there's no violation proceed with the next steps. Similar to earlier method it's important to check if the source and the target both have the same endian.
Put the user defined tablespace to read only mode and run the export. The parameter file content is shown below.

cat tts.par
dumpfile=tts114.dmp
logfile=tts114.log
directory=EXEC_DIR
transport_tablespaces=EXAMPLE,NEWTBS,SQLT,TBS2K,TBS32K,TEST,USERS
transport_full_check=y

expdp system parfile=tts.par

End of the export the output log will list the data files for transport

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
Starting "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01":  system/******** parfile=tts.par
Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/TABLE
Processing object type TRANSPORTABLE_EXPORT/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type TRANSPORTABLE_EXPORT/INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/INDEX/FUNCTIONAL_INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/REF_CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/TABLE_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
Master table "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01" successfully loaded/unloaded
******************************************************************************
Dump file set for SYSTEM.SYS_EXPORT_TRANSPORTABLE_01 is:
  /usr/local/exdata/tts114.dmp
******************************************************************************
Datafiles required for transportable tablespace EXAMPLE:
  +DATA/ent11g2/datafile/example.268.785857285
Datafiles required for transportable tablespace NEWTBS:
  +DATA/ent11g2/datafile/newtbs.273.888232727
Datafiles required for transportable tablespace SQLT:
  +DATA/ent11g2/datafile/sqlt.269.826463789
Datafiles required for transportable tablespace TBS2K:
  +DATA/ent11g2/datafile/tbs2k.272.921767965
Datafiles required for transportable tablespace TBS32K:
  +DATA/ent11g2/datafile/tbs32k.271.921768047
Datafiles required for transportable tablespace TEST:
  +DATA/ent11g2/datafile/test.274.888250337
Datafiles required for transportable tablespace USERS:
  +DATA/ent11g2/datafile/users.264.785694801
Job "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01" successfully completed at Mon Sep 5 16:36:07 2016 elapsed 0 00:01:41

Same as previous method copy the data files out of ASM. Once copied out put the tablespace to read write mode. Transfer the data files copied out of ASM to PDB's data file location (same as earlier method).
On the PDB create a DB directory and transfer the exported dump file. Unlike the TDB method, under TTS method before the import the users must exist in the PDB. If not import will fail

ORA-39123: Data Pump transportable tablespace job aborted
ORA-29342: user ASANGA does not exist in the database

Once all requried users are created in the PDB begin the import. The content of the import parameter file is given below. The TRANSPORT_DATAFILES reflect the actual file paths of the data files.

cat ttsimp.par
DUMPFILE=tts114.dmp
DIRECTORY=tts_dir
TRANSPORT_DATAFILES=
'/opt/app/oracle/oradata/CXCDB/3BC5DE03FC5E720DE0536300A8C07F97/datafile/EXAMPLE.268.785857285',
'/opt/app/oracle/oradata/CXCDB/3BC5DE03FC5E720DE0536300A8C07F97/datafile/NEWTBS.273.888232727',
'/opt/app/oracle/oradata/CXCDB/3BC5DE03FC5E720DE0536300A8C07F97/datafile/SQLT.269.826463789',
'/opt/app/oracle/oradata/CXCDB/3BC5DE03FC5E720DE0536300A8C07F97/datafile/TBS2K.272.921767965',
'/opt/app/oracle/oradata/CXCDB/3BC5DE03FC5E720DE0536300A8C07F97/datafile/TBS32K.271.921768047',
'/opt/app/oracle/oradata/CXCDB/3BC5DE03FC5E720DE0536300A8C07F97/datafile/TEST.274.888250337',
'/opt/app/oracle/oradata/CXCDB/3BC5DE03FC5E720DE0536300A8C07F97/datafile/USERS.264.785694801'
LOGFILE=tts114imp.log

impdp system@pdb114 parfile=ttsimp.par

After the import the tbalespaces will be plugged but read only mode.

SQL>  SELECT tablespace_name, plugged_in, status FROM   dba_tablespaces;

TABLESPACE_NAME                PLU STATUS
------------------------------ --- ---------
SYSTEM                         NO  ONLINE
SYSAUX                         NO  ONLINE
TEMP                           NO  ONLINE
EXAMPLE                        YES READ ONLY
NEWTBS                         YES READ ONLY
SQLT                           YES READ ONLY
TBS2K                          YES READ ONLY
TBS32K                         YES READ ONLY
TEST                           YES READ ONLY
USERS                          YES READ ONLY

Change the read only mode to read write

alter tablespace EXAMPLE read write;
alter tablespace NEWTBS read write;
alter tablespace SQLT read write;
alter tablespace TBS2K read write;
alter tablespace TBS32K read write;
alter tablespace TEST read write;
alter tablespace USERS read write;

SQL>  SELECT tablespace_name, plugged_in, status FROM   dba_tablespaces;

TABLESPACE_NAME                PLU STATUS
------------------------------ --- ---------
SYSTEM                         NO  ONLINE
SYSAUX                         NO  ONLINE
TEMP                           NO  ONLINE
EXAMPLE                        YES ONLINE
NEWTBS                         YES ONLINE
SQLT                           YES ONLINE
TBS2K                          YES ONLINE
TBS32K                         YES ONLINE
TEST                           YES ONLINE
USERS                          YES ONLINE

At this stage the PDB is ready for use with pre-12c DB plugged in.

Related Posts
Remote Cloning of a PDB
Move a PDB Between Servers
Plugging a SE2 non-CDB as an EE PDB Using File Copying and Remote Link

↧

ORA-39127: unexpected error from call to "SYS"."DBMS_JVM_EXP_PERMS"."GRANT_SYSPRIVS_EXP"

December 1, 2016, 12:00 am

≫ Next: ASMCMD cp Fails on CIFS

≪ Previous: Plugging a Non-CDB (pre-12c) into CDB Using Transportable DB (TDB) / Tablespaces (TTS)

While doing an export (expdp) following error was encountered.

Processing object type DATABASE_EXPORT/ROLE
ORA-39127: unexpected error from call to "SYS"."DBMS_JVM_EXP_PERMS"."GRANT_SYSPRIVS_EXP"
ORA-44002: invalid object name
ORA-06512: at "SYS.DBMS_ASSERT", line 383
ORA-06512: at "SYS.DBMS_METADATA", line 9408

The reason for this is the java components are removed but DBMS_JVM_EXP_PERMS package remains in the database.

SQL> select comp_name,status from dba_registry order by 2;

COMP_NAME                                STATUS
---------------------------------------- --------
Oracle Database Java Packages            REMOVED
JServer JAVA Virtual Machine             REMOVED
Oracle XDK                               REMOVED
Oracle Real Application Clusters         VALID
Oracle Database Catalog Views            VALID
Oracle Workspace Manager                 VALID
Oracle XML Database                      VALID
Oracle Database Packages and Types       VALID

To fix this, remove the DBMS_JVM_EXP_PERMS package from the database. Refer 1095533.1 for exact steps. Removing this package doesn't make any changes to component status. Java components will be shown as removed. However the packages left behind during the removal process is cleaned up.
Afterwards expdp continue without any issue.

Useful metalink note
DataPump Export (EXPDP) Failed On Identifier SYS.DBMS_JVM_EXP_PERMS Must Be Declared [ID 1095533.1]

Related Posts
ORA-39127: unexpected error from call to export_string :=WMSYS.LT_EXPORT_PKG.SCHEMA_INFO_EXP
ORA-39127: unexpected error from call to export_string :=SYS.DBMS_CUBE_EXP.SCHEMA_INFO_EXP while Exporting

↧

ASMCMD cp Fails on CIFS

January 17, 2017, 2:48 am

≫ Next: First Look at 12.2 (12.2.0.1) Installation, CDB & PDB Creation

≪ Previous: ORA-39127: unexpected error from call to "SYS"."DBMS_JVM_EXP_PERMS"."GRANT_SYSPRIVS_EXP"

Copying a file using asmcmd cp to a CIFS location fails as follows

ASMCMD> cp annnf0_ARCHIVE_LOG_0.382.923545643 /mnt/dbbackup
copying +flash/livedb/backupset/2016_09_26/annnf0_ARCHIVE_LOG_0.382.923545643 -> /mnt/dbbackup/annnf0_ARCHIVE_LOG_0.382.923545643
ASMCMD-08016: copy source->'+flash/livedb/backupset/2016_09_26/annnf0_ARCHIVE_LOG_0.382.923545643' and target->'/mnt/dbbackup/annnf0_ARCHIVE_LOG_0.382.923545643' failed
ORA-19510: failed to set size of 173851 blocks for file "/mnt/dbbackup/annnf0_ARCHIVE_LOG_0.382.923545643" (block size=512)
ORA-27045: unable to close the file
Linux-x86_64 Error: 5: Input/output error
Additional information: 4
ORA-06512: at "SYS.X$DBMS_DISKGROUP", line 258
ORA-06512: at line 3 (DBD ERROR: OCIStmtExecute)

However if the copied location is examined the file is there with same size as source. However integrity of the file cannot be guaranteed as cp command has failed. The error message suggests "unable to close the file" so it could be that file is copied but file descriptor could not be closed. Copying to other file systems (ext3) worked fine.

ASMCMD> cp annnf0_ARCHIVE_LOG_0.382.923545643 /home/oracle
copying +flash/livedb/backupset/2016_09_26/annnf0_ARCHIVE_LOG_0.382.923545643 -> /home/oracle/annnf0_ARCHIVE_LOG_0.382.923545643

So issue was assumed not to be with asmcmd but the file system.
Trying to create a backup on the CIFS location also failed

RMAN> backup current controlfile format '/mnt/dbbackup/controlc.tl';

Starting backup at 28-Sep-2016 09:14:12
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=1029 instance=livedb2 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including current control file in backup set
channel ORA_DISK_1: starting piece 1 at 28-Sep-2016 09:14:15
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 09/28/2016 09:14:16
ORA-19504: failed to create file "/mnt/dbbackup/controlc.tl"
ORA-27040: file create error, unable to create file
Linux-x86_64 Error: 22: Invalid argument

Looking in MOS several notes were found which suggest limitations of using certain Oracle utilities with CIFS. For backups it was suggested to change the filesystemio_options to asynch or failing that to mount the CIFS location with file type cifs and oracle user's user id , group id

mount -t cifs -o username=oracle,uid=oracle,gid=oinstall //cfis location /mount point

Second solution had already been implemented and asmcmd cp and rman backup still failed. However after changing filesystemio_options to asynch, rman backups succeded but asmcmd cp continue to fail. It seems asmcmd is another utility that cannot be used with CIFS (disclaimer: could not find any official oracle document to verify this though).
Beside asmcmd , datapump is also not supported on CIFS (793805.1). CIFS could be used for RMAN backups but it is not a certified file system. Which means if there's any issue when using RMAN with CIFS there won't be any oracle support to resolve the issue (444809.1).

Useful Metalink Notes
Data Pump Fails With Errors ORA-39097 ORA-39065 ORA-31643 ORA-19510 Using CIFS [793805.1]
Is use of CIFS Protocol for RMAN backups supported? [444809.1]
'cp' Fails with 'cp: cannot create symbolic link' When Copying Files to CIFS Filesystem [728223.1]
ORA-19504 ORA-27040 During Rman Backup To A Mounted Windows Directory In Linux [464267.1]
Which are the mount settings for SAMBA? [557057.1]
Product Support Document for Common Internet File System (CIFS)[1398067.1]

How to mount Windows share on Red Hat Enterprise Linux system using CIFS?

↧

First Look at 12.2 (12.2.0.1) Installation, CDB & PDB Creation

March 15, 2017, 9:33 am

≫ Next: Upgrading Oracle Single Instance with ASM (Oracle Restart) from 12.1.0.2 to 12.2.0.1

≪ Previous: ASMCMD cp Fails on CIFS

Oracle database 12.2 (12.2.0.1) for on-premises is now available. This is an brief look at the changes on 12.2 compared to previous version of 12c. Full list of changes (new, depreciate and desupported) is available here.
One of the first thing is RHEL5/OEL5 kernels are no longer supported (see supported kernels). 12.1 had support for OEL5 and RHEL5. Therefore any upgrade plan must consider the OS support and if the current DB is running on an unsupported kernel, then OS must be upgraded before DB upgrade.
There's not much difference in the installation (software only) process. However, at the end there's an option to install trace file analyzer. This was previously prompted only for grid installation with 12.2 its available for single instance installation as well.

Do you want to setup Oracle Trace File Analyzer (TFA) now ? yes|[no] :
yes
Installing Oracle Trace File Analyzer (TFA).
Log File: /opt/app/oracle/product/12.2.0/dbhome_1/install/root_hpc6.codegen.net_2017-03-15_15-46-55-462047607.log
Finished installing Oracle Trace File Analyzer (TFA)

Following images shows the changes in the DBUA.
12.2 has new feature which allow local undo tablespaces for pdbs. Following from Admin guide"CDB undo mode - A CDB can run in local undo mode or shared undo mode. Local undo mode means that every container in the CDB uses local undo. Shared undo mode means that there is one active undo tablespace for a single-instance CDB. For an Oracle RAC CDB, there is one active undo tablespace for each instance for each PDB in local undo mode." DBUA gives an option to select this mode at the creation time. It is also possible to change the undo tablespace to local after the creation of the CDB.
It appears that non-CDBs could be created with 12.2 as well. However according to upgrade guide"The non-CDB architecture was deprecated in Oracle Database 12c. It can be desupported and unavailable in a release after Oracle Database 12c Release 2. Oracle recommends use of the CDB architecture."

It is now possible to remove DB components that are not required, to keep inline with the license policies and avoiding accidental use of features not licensed for. This was not possible with 12.1 especially when CDBs were created and manual removal of these options was the only way available(refer 2001512.1).

Memory management option defaults to automatic shared memory management.

The default process count is set at 640, increased from 300.

The default character set option is Unicode AL32UTF8. Previously this would default to OS specific language setting (WE8MSWIN1252).

If a DB template is created, with 12.2 it is possible to specify a custom location for the template file.

All initialization parameters dialog box allows addition of extra parameters to SPfile.

Summary page (compare with previous version)

The progress page does not have the activity log and alert log buttons to view the log files.

End of the CDB creation following components are installed.

SQL> select comp_name,status from dba_registry;

COMP_NAME                                STATUS
---------------------------------------- --------------------------------------------
Oracle Database Catalog Views            VALID
Oracle Database Packages and Types       VALID
Oracle Real Application Clusters         OPTION OFF
Oracle XML Database                      VALID
Oracle Workspace Manager                 VALID
Oracle Text                              VALID

6 rows selected.

Next is the creation of PDB. DBCA now allows OS based authentication when connecting to CDB to create PDBs. If OS based authentication is disabled then password based authentication is needed.

Option of creating a clone is added in addition to previously available options.

Option to lock existing users and specify a new admin user.

The PDB's data file location is determined by CDB's file location. In this case CDB is on OMF as such the PDB data files will be placed in a relative path to CDB and cannot be changed at creation time.

Summary

Once the PDB is created it will be available on read write mode

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB122                         READ WRITE NO

Datafile location

SQL> select con_id,file_name from cdb_data_files order by 1;

    CON_ID FILE_NAME
---------- --------------------------------------------------------------------------------------------------------------
         1 /opt/app/oracle/oradata/CDB122/datafile/o1_mf_system_ddlt81f7_.dbf
         1 /opt/app/oracle/oradata/CDB122/datafile/o1_mf_users_ddltbf4d_.dbf
         1 /opt/app/oracle/oradata/CDB122/datafile/o1_mf_undotbs1_ddlt8wow_.dbf
         1 /opt/app/oracle/oradata/CDB122/datafile/o1_mf_sysaux_ddlt8k2f_.dbf
         4 /opt/app/oracle/oradata/CDB122/4AC96F423E65320FE0536800A8C0EC46/datafile/o1_mf_system_ddlzp0jv_.dbf
         4 /opt/app/oracle/oradata/CDB122/4AC96F423E65320FE0536800A8C0EC46/datafile/o1_mf_users_ddlzp0k3_.dbf
         4 /opt/app/oracle/oradata/CDB122/4AC96F423E65320FE0536800A8C0EC46/datafile/o1_mf_undotbs1_ddlzp0k2_.dbf
         4 /opt/app/oracle/oradata/CDB122/4AC96F423E65320FE0536800A8C0EC46/datafile/o1_mf_sysaux_ddlzp0k0_.dbf

8 rows selected.

This has been a brief look at installing and creating CDB and PDB with 12.2.

Related Posts
Installing 12c (12.1.0.1) RAC on RHEL 6 with Role Seperation - Creating CDB & PDB

↧

Upgrading Oracle Single Instance with ASM (Oracle Restart) from 12.1.0.2 to 12.2.0.1

April 4, 2017, 8:31 am

≫ Next: Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Clusterware

≪ Previous: First Look at 12.2 (12.2.0.1) Installation, CDB & PDB Creation

This post lists the steps for upgrading an Oracle restart environment (Single instance non-CDB on ASM) from 12.1.0.2 to 12.2.0.1. The 12.1 setup was on RHEL 7 and had a OS upgrade from RHEL 7.0 to RHEL 7.3. The current versions of the GI are

[grid@rhel7 ~]$ crsctl query has releaseversion
Oracle High Availability Services release version on the local node is [12.1.0.2.0]
[grid@rhel7 ~]$ crsctl query has softwareversion
Oracle High Availability Services version on the local node is [12.1.0.2.0]

The Oracle restart environment used is a role separate setup where GI is installed as grid user while Oracle software is installed Oracle user. The GI and Oracle homes had the latest PSU applied (Database Patch Set Update : 12.1.0.2.170117). However orachk pre-upgrade listed following patches as missing.

./orachk -u -o pre
...
 WARNING => Oracle patch 21856522 is not applied on RDBMS_HOME /opt/app/oracle/product/12.1.0/dbhome_2
 WARNING => Oracle patch 20958816 is not applied on RDBMS_HOME /opt/app/oracle/product/12.1.0/dbhome_2
...

MOS note 2180188.1 list patches to apply before upgrading to 12.2. Cluvfy could be used to further check the system setup

./runcluvfy.sh stage -pre hacfg

This option is not for upgrade check but could be used to check readiness of oracle restart installation.
Unlike the previous versions, 12.2 grid infrastructure requires unzipping it into a grid home. There's no GI home selection on the OUI. Create the new directory path for 12.2 GI and unzip the GI bundle.

mkdir -p /opt/app/oracle/product/12.2.0/grid
cp ~/linuxx64_12201_grid_home.zip /opt/app/oracle/product/12.2.0/grid
cd /opt/app/oracle/product/12.2.0/grid
unzip linuxx64_12201_grid_home.zip

ASM will be upgraded as part of the upgrade process. Therefore stop the database before starting the GI upgrade.

srvctl stop database -d se2db

Start the GI upgrade by running the gridSetup.sh from the grid home.

grid@rhel7 grid]$ ./gridSetup.sh

This will start the GI installer and select the upgrade GI option.

As mentioned earlier, the installer prompts to shutdown the DB as ASM is upgraded.

If wanted GI could be registered with a cloud control.

OS groups for authentication into ASM. These are same as 12.1 setup.

Oracle base location could be selected but not the GI Home location. This will be fixed, based on where the GI software is extracted.

Running of root scripts could be automated by providing the root password or user with sudo privileges.

Prerequisite check. The minimum memory requirement is 8GB. As this is a test setup this requirement is ignored and proceeded to next step.

Summary page and progress.

Depending on the option selected, (either to automatically run root scripts or manually run the root scripts) one of the following options will be presented.

In either case (auto run or manual run) the root upgrade script failed due to ACFS related known issue. Output below shows when rootupgrade.sh was run manually.

[root@rhel7 grid]# ./rootupgrade.sh
Performing root user operation.

The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /opt/app/oracle/product/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to /usr/local/bin ...
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/app/oracle/product/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/opt/app/oracle/crsdata/rhel7/crsconfig/roothas_2017-04-03_05-38-44PM.log
2017/04/03 17:38:45 CLSRSC-363: User ignored prerequisites during installation

ASM has been upgraded and started successfully.

Creating OCR keys for user 'grid', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node rhel7 successfully pinned.
2017/04/03 17:39:32 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'

CRS-4123: Oracle High Availability Services has been started.

2017/04/03 17:58:46 CLSRSC-482: Running command: 'srvctl upgrade model -s 12.1.0.2.0 -d 12.2.0.1.0 -p first'
2017/04/03 17:58:51 CLSRSC-482: Running command: 'srvctl upgrade model -s 12.1.0.2.0 -d 12.2.0.1.0 -p last'

rhel7 2017/04/03 17:58:54 /opt/app/oracle/product/12.2.0/grid/cdata/rhel7/backup_20170403_175854.olr 0

rhel7 2015/09/29 16:40:16 /opt/app/oracle/product/12.1.0/grid/cdata/rhel7/backup_20150929_164016.olr 0
2017/04/03 17:59:27 CLSRSC-400: A system reboot is required to continue installing.
The command '/opt/app/oracle/product/12.2.0/grid/perl/bin/perl -I/opt/app/oracle/product/12.2.0/grid/perl/lib -I/opt/app/oracle/product/12.2.0/grid/crs/install
/opt/app/oracle/product/12.2.0/grid/crs/install/roothas.pl -upgrade' execution failed

Looking at the log file it could be seen the error happens related to ACFS

2017-04-03 17:59:26: /opt/app/oracle/product/12.2.0/grid/bin/crsctl disable has ... disabling CRS in preparation for reboot.
2017-04-03 17:59:26: Executing cmd: /opt/app/oracle/product/12.2.0/grid/bin/crsctl disable has
2017-04-03 17:59:27: Command output:
>  CRS-4621: Oracle High Availability Services autostart is disabled.
>End Command output
2017-04-03 17:59:27: /opt/app/oracle/product/12.2.0/grid/bin/crsctl disable has ... CRS disabled, ready for reboot.
2017-04-03 17:59:27: USM driver install status is 3
2017-04-03 17:59:27: ACFS drivers unable to be installed.
2017-04-03 17:59:27: Executing cmd: /opt/app/oracle/product/12.2.0/grid/bin/clsecho -p has -f clsrsc -m 400
2017-04-03 17:59:27: Command output:
> CLSRSC-400: A system reboot is required to continue installing.
>End Command output
2017-04-03 17:59:27: CLSRSC-400: A system reboot is required to continue installing.

According to MOS note 2025056.1 this issue could be ignored if ACFS is not used or not supported. Rootupgrade scripts are restartable when upgrading to 11.2.0.2 or above (1364947.1). During the subsequent execute of the rootupgrade.sh the script executes without any issue.

[root@rhel7 grid]# ./rootupgrade.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /opt/app/oracle/product/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/app/oracle/product/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /opt/app/oracle/crsdata/rhel7/crsconfig/roothas_2017-04-03_06-09-16PM.log
2017/04/03 18:09:16 CLSRSC-363: User ignored prerequisites during installation
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel7'
CRS-2673: Attempting to stop 'ora.evmd' on 'rhel7'
CRS-2677: Stop of 'ora.evmd' on 'rhel7' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel7' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/04/03 18:10:35 CLSRSC-327: Successfully configured Oracle Restart for a standalone server

When the root script failure happens during auto run of the root script, click retry button to run the root script again. It was found that after the initial failure, the subsequent execution complete successfully.

The GI software versions are now upgraded to 12.2

[grid@rhel7 ~]$ crsctl query has releaseversion
Oracle High Availability Services release version on the local node is [12.2.0.1.0]
[grid@rhel7 ~]$ crsctl query has softwareversion
Oracle High Availability Services version on the local node is [12.2.0.1.0]

Cluvfy could be used for post upgrade verification.

cluvfy stage -post hacfg

Verifying Oracle Restart Integrity ...PASSED
Verifying OLR Integrity ...PASSED

Post-check for Oracle Restart configuration was successful.

CVU operation performed:      stage -post hacfg
Date:                         04-Apr-2017 13:05:07
CVU home:                     /opt/app/oracle/product/12.2.0/grid/
User:                         grid

The next step is the upgrade of database software. This is done by doing a out-of-place software only installation of the 12.2 DB software. One new thing introduced in 12.2 is the OS group specific for RAC administration. Following from install guide "You must designate a group as the OSRACDBA group during database installation. Members of this group are granted the SYSRAC privileges to perform day–to–day administration of Oracle databases on an Oracle RAC cluster". This user group could be used for oracle restart as well (This does lead to an issue which is mentioned towards the end of the post). In order to use this OS group, create it as root and assign it to oracle user.

[root@rhel7 oracle]# groupadd racdba
[root@rhel7 oracle]# usermod -g oinstall -G dba,oper,asmoper,asmdba,asmadmin,backupdba,dgdba,kmdba,racdba oracle
[root@rhel7 oracle]# id oracle
uid=1001(oracle) gid=1001(oinstall) groups=1001(oinstall),1002(dba),1003(oper),1005(asmoper),1006(asmdba),1007(asmadmin),1008(backupdba),1009(dgdba),1010(kmdba),1011(racdba)

During the software installation select this group for RAC administration.

Summary

Once the DB software is installed, the last step is to upgrade the database. 12.2 provide a new pre-upgrade check tool which comes in the form of a jar file. (12.2_Oracle_Home/rdbms/admin/preupgrade.jar). This could be used to verify the pre-upgrade status of the database. Before running the tool set the Oracle home, oracle base and oracle sid and path environment variables. Below is the output of running the pre-upgrade tool.

[oracle@rhel7 ~]$ $ORACLE_HOME/jdk/bin/java -jar preupgrade.jar TERMINAL
Report generated by Oracle Database Pre-Upgrade Information Tool Version
12.2.0.1.0

Upgrade-To version: 12.2.0.1.0

=======================================
Status of the database prior to upgrade
=======================================

      Database Name:  SE2DB
     Container Name:  SE2DB
       Container ID:  0
            Version:  12.1.0.2.0
         Compatible:  12.1.0.2.0
          Blocksize:  8192
           Platform:  Linux x86 64-bit
      Timezone File:  18
  Database log mode:  ARCHIVELOG
           Readonly:  FALSE
            Edition:  SE

  Oracle Component                       Upgrade Action    Current Status
  ----------------                       --------------    --------------
  Oracle Server                          [to be upgraded]  VALID
  Oracle Workspace Manager               [to be upgraded]  VALID
  Oracle Text                            [to be upgraded]  VALID
  Oracle XML Database                    [to be upgraded]  VALID


==============
BEFORE UPGRADE
==============

  Run /preupgrade_fixups.sql to complete all
  of the BEFORE UPGRADE action items below marked with '(AUTOFIXUP)'.

  REQUIRED ACTIONS
  ================
   + Adjust TABLESPACE SIZES as needed.
                                                Auto      12.2.0.1.0
     Tablespace                        Size     Extend    Min Size    Action
     ----------                     ----------  --------  ----------  ------

     SYSAUX                             550 MB  DISABLED      500 MB  None
     SYSTEM                             700 MB  ENABLED       797 MB  None
     TEMP                                20 MB  ENABLED       150 MB  None
     UNDOTBS1                           200 MB  ENABLED       400 MB  None

     Note that 12.2.0.1.0 minimum sizes are estimates.
     If you plan to upgrade multiple pluggable databases concurrently,
     then you must ensure that the UNDO tablespace size is equal to at least
     the number of pluggable databases that you upgrade concurrently,
     multiplied by that minimum.  Failing to allocate sufficient space can
     cause the upgrade to fail.

  RECOMMENDED ACTIONS
  ===================
   + Run 12.1.0.2.0 $ORACLE_HOME/rdbms/admin/utlrp.sql to recompile invalid
     objects.  You can view the individual invalid objects with

       SET SERVEROUTPUT ON;
       EXECUTE DBMS_PREUP.INVALID_OBJECTS;

     5 objects are INVALID.

     There should be no INVALID objects in SYS/SYSTEM or user schemas before
     database upgrade.

   + (AUTOFIXUP) Gather stale data dictionary statistics prior to database
     upgrade in off-peak time using:

      EXECUTE DBMS_STATS.GATHER_DICTIONARY_STATS;

     Dictionary statistics do not exist or are stale (not up-to-date).

     Dictionary statistics help the Oracle optimizer find efficient SQL
     execution plans and are essential for proper upgrade timing. Oracle
     recommends gathering dictionary statistics in the last 24 hours before
     database upgrade.

     For information on managing optimizer statistics, refer to the 12.1.0.2
     Oracle Database SQL Tuning Guide.

=============
AFTER UPGRADE
=============

  Run /postupgrade_fixups.sql to complete all
  of the AFTER UPGRADE action items below marked with '(AUTOFIXUP)'.

  REQUIRED ACTIONS
  ================
  None

  RECOMMENDED ACTIONS
  ===================
   + Upgrade the database time zone version using the DBMS_DST package.

     The database is using timezone datafile version 18 and the target
     12.2.0.1.0 database ships with timezone datafile version 26.

     Oracle recommends using the most recent timezone data.  For further
     information, refer to My Oracle Support Note 1585343.1.

   + (AUTOFIXUP) Gather dictionary statistics after the upgrade using the
     command:

       EXECUTE DBMS_STATS.GATHER_DICTIONARY_STATS;

     Oracle recommends gathering dictionary statistics after upgrade.

     Dictionary statistics provide essential information to the Oracle
     optimizer to help it find efficient SQL execution plans. After a
     database upgrade, statistics need to be re-gathered as there can now be
     tables that have significantly changed during the upgrade or new tables
     that do not have statistics gathered yet.

Preupgrade generated files:
    /opt/app/oracle/cfgtoollogs/se2db/preupgrade/preupgrade_fixups.sql
    /opt/app/oracle/cfgtoollogs/se2db/preupgrade/postupgrade_fixups.sql

Run the preupgrade_fixups.sql script to rectify any pre-upgrade issues raised. The timezone filed used by 12.1 is 18 and this will be upgraded to 26 as part of the DB upgrade

SQL> select * from v$timezone_file;

FILENAME                VERSION     CON_ID
-------------------- ---------- ----------
timezlrg_18.dat              18          0

Run the DBUA from 12.2 home to begin the database upgrade.

Upgrade Summary

Upgrade Results

Once the DBUA completes the upgrade run the post upgrade fix script, mentioned by the pre-upgrade tool

/@opt/app/oracle/cfgtoollogs/se2db/preupgrade/postupgrade_fixups.sql

Executing Oracle POST-Upgrade Fixup Script

Auto-Generated by:       Oracle Preupgrade Script
                         Version: 12.2.0.1.0 Build: 1
Generated on:            2017-03-31 18:27:51

For Source Database:     SE2DB
Source Database Version: 12.1.0.2.0
For Upgrade to Version:  12.2.0.1.0

                          Fixup
Check Name                Status  Further DBA Action
----------                ------  ------------------
old_time_zones_exist      Passed  None
post_dictionary           Passed  None

PL/SQL procedure successfully completed.

Verify the timezone file has been updated

SQL> select * from v$timezone_file;

FILENAME                VERSION     CON_ID
-------------------- ---------- ----------
timezlrg_26.dat              26          0

After the upgrade using srvctl to administer the database will result in error as shown below.

[oracle@rhel7 ~]$ srvctl stop database -db se2db
PRCD-1334 : failed to stop database se2db
PRCD-1124 : Failed to stop database se2db and its services
PRCD-1131 : Failed to stop database se2db and its services on nodes
PRCR-1133 : Failed to stop database se2db and its running services
PRCR-1132 : Failed to stop resources using a filter
ORA-01017: invalid username/password; logon denied
CRS-2675: Stop of 'ora.se2db.db' on 'rhel7' failed

The reason for this is the rac admin OS group that was created during database software installation. Oracle document only mentions of making grid part of sysdba group. Following from Oracle install guide "When you manually add a database to the Oracle Restart configuration, you must also add the Oracle grid infrastructure software owner as a member of the OSDBA group of that database. This is because the grid infrastructure components must be able to connect to the database as SYSDBA to start and stop the database. For example, if the host user who installed the grid infrastructure home is named grid and the OSDBA group of the new database is named dba, then user grid must be a member of the dba group". However if a separate OS group is ued for RAC administration then grid user must be part of that group as well to prevent errors as above.

[root@rhel7 ~]# id grid
uid=1002(grid) gid=1001(oinstall) groups=1001(oinstall),1002(dba),1005(asmoper),1006(asmdba),1007(asmadmin)
[root@rhel7 ~]# usermod -g oinstall -G dba,asmoper,asmdba,asmadmin,racdba grid
[root@rhel7 ~]# id grid
uid=1002(grid) gid=1001(oinstall) groups=1001(oinstall),1002(dba),1005(asmoper),1006(asmdba),1007(asmadmin),1011(racdba)

After this chagne srvctl could be usd to manged the DB without any issue.
If satisfied with the upgrade and application testing change the compatibility parameter on the DB and ASM disk groups.

SQL> alter system set compatible='12.2.0.1.0' scope=spfile;
shutdown immediate;

SQL>  alter diskgroup FRA SET attribute 'compatible.asm'='12.2.0.1.0';
SQL>  alter diskgroup DATA  SET attribute 'compatible.asm'='12.2.0.1.0';
SQL> alter diskgroup fra set attribute 'compatible.rdbms'='12.2.0.1.0';
SQL>  alter diskgroup data set attribute 'compatible.rdbms'='12.2.0.1.0';

SQL>  select g.name,a.name,a.value from v$asm_diskgroup g, v$asm_attribute a where g.group_number=a.group_number and a.name like '%compat%';

NAME                                     NAME                                     VALUE
---------------------------------------- ---------------------------------------- ----------------------------------------
DATA                                     compatible.asm                           12.2.0.1.0
DATA                                     compatible.rdbms                         12.2.0.1.0
FRA                                      compatible.asm                           12.2.0.1.0
FRA                                      compatible.rdbms                         12.2.0.1.0

Start the database and run orachk post ugprade check and cluvfy post checks

./orachk -u -o post

 cluvfy stage -post hacfg

Verifying Oracle Restart Integrity ...PASSED
Verifying OLR Integrity ...PASSED

Post-check for Oracle Restart configuration was successful.

CVU operation performed:      stage -post hacfg
Date:                         04-Apr-2017 13:05:07
CVU home:                     /opt/app/oracle/product/12.2.0/grid/
User:                         grid

Useful metalink notes
ACFS Drivers Install reports CLSRSC-400: A system reboot is required to continue installing [ID 2025056.1]
Patches to apply before upgrading Oracle GI and DB to 12.2.0.1 [ID 2180188.1]
12.2 Grid Infrastructure Installation: What's New [ID 2024946.1]
How to Upgrade to/Downgrade from Grid Infrastructure 12.2 and Known Issues [ID 2240959.1]

Related Posts
Upgrading Single Instance on ASM from 11.2.0.3 to 11.2.0.4
Upgrading Grid Infrastructure Used for Single Instance from 11.2.0.4 to 12.1.0.2

↧

Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Clusterware

May 25, 2017, 9:18 am

≫ Next: Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Database

≪ Previous: Upgrading Oracle Single Instance with ASM (Oracle Restart) from 12.1.0.2 to 12.2.0.1

This post is not a step by step guide to installing 12cR2 RAC but list the main differences in the installation process compared to previous versions and things to look out for. One of the main difference in installing 12cR2 clusterware is that, it is no longer done through runInstaller. The downloaded Oracle Grid Infrastructure image files are unzipped into the folder path where GI is to be installed. This only need to be done on the local node only. During installation, the software is copied and installed on all other nodes in the cluster.
If it is planned to ASM filter driver (ASMFD) for managing disks used by ASM then uninstall ASMLib if it is already installed.
Unlike 12cR1, it is possible to specify a separate disk group for GIMR at the installation time. Oracle documentation says "You cannot migrate the GIMR from one disk group to another later". 12cR1 also mentioned the same but there were ways to move GIMR to different disk group after the installation. It has been not verified whether the same techniques could be used for 12.2 as well.
1. Shown below is the OS and kernel versions used for this RAC installation. These are the minimum versions supported for 12.2 clusterware installation. If the OS/Kernel versions are lower than this version then OS need to be upgraded before installation could proceed.

uname -r
2.6.32-358.el6.x86_64
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.4 (Santiago)

2. Create the user groups and users similar to previous 12.1 installation. As said in the previous post 12.2 introduce a new user group for RAC management. Following from install guide "You must designate a group as the OSRACDBA group during database installation. Members of this group are granted the SYSRAC privileges to perform day–to–day administration of Oracle databases on an Oracle RAC cluster".

groupadd dba
groupadd oinstall
groupadd oper
groupadd asmoper
groupadd asmdba
groupadd asmadmin
groupadd backupdba
groupadd dgdba
groupadd kmdba
groupadd racdba

useradd -g oinstall -G dba,oper,asmdba,asmoper,asmadmin,backupdba,dgdba,kmdba,racdba oracle
useradd -g oinstall -G asmadmin,asmdba,asmoper,dba grid

3. The other pre-requisite steps are similar to 12.1 RAC installation. Follow oracle installation guide for detail instructions. As said earlier the grid infrastructure software is unzip directly into the install location.

mkdir -p /opt/app/12.2.0/grid
cp ~/linuxx64_12201_grid_home.zip /opt/app/12.2.0/grid
cd /opt/app/12.2.0/grid
unzip linuxx64_12201_grid_home.zip

4. 12.2 gives the option of using ASM filter driver (ASMFD) for managing disks used by ASM. However installation could be done without the use of AFD as well. Using udev rules for the ASM disks. For this setup UDEV rules have been setup as below. The disks used for OCR were of 3GB (OCR disk group to be of normal redundancy) and GIMR disk group is 40GB (with external redundancy). Minimum storage requirement for standaloen cluster could be found here.

KERNEL=="sd*1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB34650f33-5ecdc029",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME="oracleasm/data1"
KERNEL=="sd*1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VBdb057b08-1733da4b",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME="oracleasm/fra1"

KERNEL=="sd*1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB1a72679c-ab23e15f",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME="oracleasm/ocr1"
KERNEL=="sd*1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB8f43e75c-3e0bec6d",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME="oracleasm/ocr2"
KERNEL=="sd*1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB1a5c8787-e082e7a7",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME="oracleasm/ocr3"

KERNEL=="sd*1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VBf8475ab7-d401ec8b",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME="oracleasm/gimr"

5. The main body of the post shows the installation without using AFD. The end of the post will have additional steps needed to be done before clusteware is install if AFD is desired. 12.2 install documentation states that gcc and gcc+ are not required on OEL and RHEL for GI installations. However the runcluvfy does look for these two packages (see output below). In this case the pre-check was unsuccessful due to lower SWAP size, which is ignorable (unless for production).

 ./runcluvfy.sh stage -pre crsinst -n rhel6m1,rhel6m2 -r 12.2 -osdba asmdba -osoper asmoper -asm -asmgrp asmadmin -presence local -asmdev /dev/oracleasm/data1,/dev/oracleasm/fra1,/dev/oracleasm/ocr1,/dev/oracleasm/ocr2,/dev/oracleasm/ocr3,/dev/oracleasm/gimr  -crshome /opt/app/12.2.0/grid -fixup  -verbose

Verifying Physical Memory ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       7.6865GB (8059836.0KB)    8GB (8388608.0KB)         passed
  rhel6m1       7.6865GB (8059836.0KB)    8GB (8388608.0KB)         passed
Verifying Physical Memory ...PASSED
Verifying Available Physical Memory ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       7.5462GB (7912808.0KB)    50MB (51200.0KB)          passed
  rhel6m1       7.2342GB (7585564.0KB)    50MB (51200.0KB)          passed
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       4GB (4194296.0KB)         7.6865GB (8059836.0KB)    failed
  rhel6m1       4GB (4194296.0KB)         7.6865GB (8059836.0KB)    failed
Verifying Swap Size ...FAILED (PRVF-7573)
Verifying Free Space: rhel6m2:/usr,rhel6m2:/var,rhel6m2:/etc,rhel6m2:/opt/app/12.2.0/grid,rhel6m2:/sbin,rhel6m2:/tmp ...
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /usr              rhel6m2       /             31.793GB      25MB          passed
  /var              rhel6m2       /             31.793GB      5MB           passed
  /etc              rhel6m2       /             31.793GB      25MB          passed
  /opt/app/12.2.0/grid  rhel6m2       /             31.793GB      6.9GB         passed
  /sbin             rhel6m2       /             31.793GB      10MB          passed
  /tmp              rhel6m2       /             31.793GB      1GB           passed
Verifying Free Space: rhel6m2:/usr,rhel6m2:/var,rhel6m2:/etc,rhel6m2:/opt/app/12.2.0/grid,rhel6m2:/sbin,rhel6m2:/tmp ...PASSED
Verifying Free Space: rhel6m1:/usr,rhel6m1:/var,rhel6m1:/etc,rhel6m1:/opt/app/12.2.0/grid,rhel6m1:/sbin,rhel6m1:/tmp ...
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /usr              rhel6m1       /             21.9208GB     25MB          passed
  /var              rhel6m1       /             21.9208GB     5MB           passed
  /etc              rhel6m1       /             21.9208GB     25MB          passed
  /opt/app/12.2.0/grid  rhel6m1       /             21.9208GB     6.9GB         passed
  /sbin             rhel6m1       /             21.9208GB     10MB          passed
  /tmp              rhel6m1       /             21.9208GB     1GB           passed
Verifying Free Space: rhel6m1:/usr,rhel6m1:/var,rhel6m1:/etc,rhel6m1:/opt/app/12.2.0/grid,rhel6m1:/sbin,rhel6m1:/tmp ...PASSED
Verifying User Existence: grid ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists(502)
  rhel6m1       passed                    exists(502)

  Verifying Users With Same UID: 502 ...PASSED
Verifying User Existence: grid ...PASSED
Verifying Group Existence: asmadmin ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists
  rhel6m1       passed                    exists
Verifying Group Existence: asmadmin ...PASSED
Verifying Group Existence: asmoper ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists
  rhel6m1       passed                    exists
Verifying Group Existence: asmoper ...PASSED
Verifying Group Existence: asmdba ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists
  rhel6m1       passed                    exists
Verifying Group Existence: asmdba ...PASSED
Verifying Group Existence: oinstall ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists
  rhel6m1       passed                    exists
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: asmdba ...
  Node Name         User Exists   Group Exists  User in Group  Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           yes           yes           yes           passed
  rhel6m1           yes           yes           yes           passed
Verifying Group Membership: asmdba ...PASSED
Verifying Group Membership: asmadmin ...
  Node Name         User Exists   Group Exists  User in Group  Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           yes           yes           yes           passed
  rhel6m1           yes           yes           yes           passed
Verifying Group Membership: asmadmin ...PASSED
Verifying Group Membership: oinstall(Primary) ...
  Node Name         User Exists   Group Exists  User in Group  Primary       Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m2           yes           yes           yes           yes           passed
  rhel6m1           yes           yes           yes           yes           passed
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Group Membership: asmoper ...
  Node Name         User Exists   Group Exists  User in Group  Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           yes           yes           yes           passed
  rhel6m1           yes           yes           yes           passed
Verifying Group Membership: asmoper ...PASSED
Verifying Run Level ...
  Node Name     run level                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       3                         3,5                       passed
  rhel6m1       3                         3,5                       passed
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           hard          65536         65536         passed
  rhel6m1           hard          65536         65536         passed
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           soft          1024          1024          passed
  rhel6m1           soft          1024          1024          passed
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           hard          16384         16384         passed
  rhel6m1           hard          16384         16384         passed
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           soft          2047          2047          passed
  rhel6m1           soft          2047          2047          passed
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           soft          10240         10240         passed
  rhel6m1           soft          10240         10240         passed
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Architecture ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       x86_64                    x86_64                    passed
  rhel6m1       x86_64                    x86_64                    passed
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       2.6.32-358.el6.x86_64     2.6.32                    passed
  rhel6m1       2.6.32-358.el6.x86_64     2.6.32                    passed
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           3010          3010          250           passed
  rhel6m2           3010          3010          250           passed
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           385280        385280        32000         passed
  rhel6m2           385280        385280        32000         passed
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           3010          3010          100           passed
  rhel6m2           3010          3010          100           passed
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           128           128           128           passed
  rhel6m2           128           128           128           passed
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           68719476736   68719476736   4126636032    passed
  rhel6m2           68719476736   68719476736   4126636032    passed
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4096          4096          4096          passed
  rhel6m2           4096          4096          4096          passed
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4294967296    4294967296    805983        passed
  rhel6m2           4294967296    4294967296    805983        passed
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           6815744       6815744       6815744       passed
  rhel6m2           6815744       6815744       6815744       passed
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           between 9000 & 65500  between 9000 & 65500  between 9000 & 65535  passed
  rhel6m2           between 9000 & 65500  between 9000 & 65500  between 9000 & 65535  passed
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4194304       4194304       262144        passed
  rhel6m2           4194304       4194304       262144        passed
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4194304       4194304       4194304       passed
  rhel6m2           4194304       4194304       4194304       passed
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           1048576       1048576       262144        passed
  rhel6m2           1048576       1048576       262144        passed
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           2097152       2097152       1048576       passed
  rhel6m2           2097152       2097152       1048576       passed
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           3145728       3145728       1048576       passed
  rhel6m2           3145728       3145728       1048576       passed
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying OS Kernel Parameter: panic_on_oops ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           1             undefined     1             passed
  rhel6m2           1             undefined     1             passed
Verifying OS Kernel Parameter: panic_on_oops ...PASSED
Verifying Package: binutils-2.20.51.0.2 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
  rhel6m1       binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
Verifying Package: binutils-2.20.51.0.2 ...PASSED
Verifying Package: compat-libcap1-1.10 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       compat-libcap1-1.10-1     compat-libcap1-1.10       passed
  rhel6m1       compat-libcap1-1.10-1     compat-libcap1-1.10       passed
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
  rhel6m1       compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...PASSED
Verifying Package: libgcc-4.4.7 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libgcc(x86_64)-4.4.7-3.el6  libgcc(x86_64)-4.4.7      passed
  rhel6m1       libgcc(x86_64)-4.4.7-3.el6  libgcc(x86_64)-4.4.7      passed
Verifying Package: libgcc-4.4.7 (x86_64) ...PASSED
Verifying Package: libstdc++-4.4.7 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libstdc++(x86_64)-4.4.7-3.el6  libstdc++(x86_64)-4.4.7   passed
  rhel6m1       libstdc++(x86_64)-4.4.7-3.el6  libstdc++(x86_64)-4.4.7   passed
Verifying Package: libstdc++-4.4.7 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.4.7 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libstdc++-devel(x86_64)-4.4.7-3.el6  libstdc++-devel(x86_64)-4.4.7  passed
  rhel6m1       libstdc++-devel(x86_64)-4.4.7-3.el6  libstdc++-devel(x86_64)-4.4.7  passed
Verifying Package: libstdc++-devel-4.4.7 (x86_64) ...PASSED
Verifying Package: sysstat-9.0.4 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
  rhel6m1       sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
Verifying Package: sysstat-9.0.4 ...PASSED
Verifying Package: gcc-4.4.7 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       gcc-4.4.7-3.el6           gcc-4.4.7                 passed
  rhel6m1       gcc-4.4.7-3.el6           gcc-4.4.7                 passed
Verifying Package: gcc-4.4.7 ...PASSED
Verifying Package: gcc-c++-4.4.7 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       gcc-c++-4.4.7-3.el6       gcc-c++-4.4.7             passed
  rhel6m1       gcc-c++-4.4.7-3.el6       gcc-c++-4.4.7             passed
Verifying Package: gcc-c++-4.4.7 ...PASSED
Verifying Package: ksh ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       ksh                       ksh                       passed
  rhel6m1       ksh                       ksh                       passed
Verifying Package: ksh ...PASSED
Verifying Package: make-3.81 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       make-3.81-20.el6          make-3.81                 passed
  rhel6m1       make-3.81-20.el6          make-3.81                 passed
Verifying Package: make-3.81 ...PASSED
Verifying Package: glibc-2.12 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       glibc(x86_64)-2.12-1.107.el6  glibc(x86_64)-2.12        passed
  rhel6m1       glibc(x86_64)-2.12-1.107.el6  glibc(x86_64)-2.12        passed
Verifying Package: glibc-2.12 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.12 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       glibc-devel(x86_64)-2.12-1.107.el6  glibc-devel(x86_64)-2.12  passed
  rhel6m1       glibc-devel(x86_64)-2.12-1.107.el6  glibc-devel(x86_64)-2.12  passed
Verifying Package: glibc-devel-2.12 (x86_64) ...PASSED
Verifying Package: libaio-0.3.107 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
  rhel6m1       libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
Verifying Package: libaio-0.3.107 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.107 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
  rhel6m1       libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
Verifying Package: libaio-devel-0.3.107 (x86_64) ...PASSED
Verifying Package: nfs-utils-1.2.3-15 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       nfs-utils-1.2.3-36.el6    nfs-utils-1.2.3-15        passed
  rhel6m1       nfs-utils-1.2.3-36.el6    nfs-utils-1.2.3-15        passed
Verifying Package: nfs-utils-1.2.3-15 ...PASSED
Verifying Package: smartmontools-5.43-1 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       smartmontools-5.43-1.el6  smartmontools-5.43-1      passed
  rhel6m1       smartmontools-5.43-1.el6  smartmontools-5.43-1      passed
Verifying Package: smartmontools-5.43-1 ...PASSED
Verifying Package: net-tools-1.60-110 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       net-tools-1.60-110.el6_2  net-tools-1.60-110        passed
  rhel6m1       net-tools-1.60-110.el6_2  net-tools-1.60-110        passed
Verifying Package: net-tools-1.60-110 ...PASSED
Verifying Port Availability for component "Oracle Notification Service (ONS)" ...
  Node Name         Port Number   Protocol      Available     Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           6200          TCP           yes           successful
  rhel6m1           6200          TCP           yes           successful
  rhel6m2           6100          TCP           yes           successful
  rhel6m1           6100          TCP           yes           successful
Verifying Port Availability for component "Oracle Notification Service (ONS)" ...PASSED
Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...
  Node Name         Port Number   Protocol      Available     Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           42424         TCP           yes           successful
  rhel6m1           42424         TCP           yes           successful
Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...PASSED
Verifying Users With Same UID: 0 ...PASSED
Verifying Current Group ID ...PASSED
Verifying Root user consistency ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m2                               passed
  rhel6m1                               passed
Verifying Root user consistency ...PASSED
Verifying Package: cvuqdisk-1.0.10-1 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       cvuqdisk-1.0.10-1         cvuqdisk-1.0.10-1         passed
  rhel6m1       cvuqdisk-1.0.10-1         cvuqdisk-1.0.10-1         passed
Verifying Package: cvuqdisk-1.0.10-1 ...PASSED
Verifying Node Connectivity ...
  Verifying Hosts File ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               passed
  rhel6m2                               passed
  Verifying Hosts File ...PASSED

Interface information for node "rhel6m2"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.94    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth1   192.168.1.88    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:69:2C:B6 1500

Interface information for node "rhel6m1"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.93    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth1   192.168.1.87    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:A3:C4:6F 1500

Check: MTU consistency of the subnet "192.168.1.0".

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth1          192.168.1.88  192.168.1.0   1500
  rhel6m1           eth1          192.168.1.87  192.168.1.0   1500

Check: MTU consistency of the subnet "192.168.0.0".

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth0          192.168.0.94  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.93  192.168.0.0   1500
  Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth1:192.168.1.87]      rhel6m2[eth1:192.168.1.88]      yes

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.94]      yes
  Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
  Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast check ...
Checking subnet "192.168.1.0" for multicast communication with multicast group "224.0.0.251"
Verifying Multicast check ...PASSED
Verifying Device Checks for ASM ...Disks "/dev/oracleasm/gimr,/dev/oracleasm/fra1,/dev/oracleasm/data1,/dev/oracleasm/ocr3,/dev/oracleasm/ocr1,/dev/oracleasm/ocr2" are managed by ASM.
Verifying Device Checks for ASM ...PASSED
Verifying I/O scheduler ...
  Verifying Package: cvuqdisk-1.0.10-1 ...PASSED
Verifying I/O scheduler ...PASSED
Verifying Network Time Protocol (NTP) ...
  Verifying '/etc/ntp.conf' ...
  Node Name                             File exists?
  ------------------------------------  ------------------------
  rhel6m2                               no
  rhel6m1                               no

  Verifying '/etc/ntp.conf' ...PASSED
  Verifying '/var/run/ntpd.pid' ...
  Node Name                             File exists?
  ------------------------------------  ------------------------
  rhel6m2                               no
  rhel6m1                               no

  Verifying '/var/run/ntpd.pid' ...PASSED
Verifying Network Time Protocol (NTP) ...PASSED
Verifying Same core file name pattern ...PASSED
Verifying User Mask ...
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       0022                      0022                      passed
  rhel6m1       0022                      0022                      passed
Verifying User Mask ...PASSED
Verifying User Not In Group "root": grid ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    does not exist
  rhel6m1       passed                    does not exist
Verifying User Not In Group "root": grid ...PASSED
Verifying Time zone consistency ...PASSED
Verifying resolv.conf Integrity ...
  Verifying (Linux) resolv.conf Integrity ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               passed
  rhel6m2                               passed

  checking response for name "rhel6m2" from each of the name servers specified
  in "/etc/resolv.conf"

  Node Name     Source                    Comment                   Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       192.168.0.66              IPv4                      passed

  checking response for name "rhel6m1" from each of the name servers specified
  in "/etc/resolv.conf"

  Node Name     Source                    Comment                   Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m1       192.168.0.66              IPv4                      passed
  Verifying (Linux) resolv.conf Integrity ...PASSED
Verifying resolv.conf Integrity ...PASSED
Verifying DNS/NIS name service ...PASSED
Verifying Domain Sockets ...PASSED
Verifying /boot mount ...PASSED
Verifying File system mount options for path GI_HOME ...PASSED
Verifying Daemon "avahi-daemon" not configured and running ...
  Node Name     Configured                Status
  ------------  ------------------------  ------------------------
  rhel6m2       no                        passed
  rhel6m1       no                        passed

  Node Name     Running?                  Status
  ------------  ------------------------  ------------------------
  rhel6m2       no                        passed
  rhel6m1       no                        passed
Verifying Daemon "avahi-daemon" not configured and running ...PASSED
Verifying Daemon "proxyt" not configured and running ...
  Node Name     Configured                Status
  ------------  ------------------------  ------------------------
  rhel6m2       no                        passed
  rhel6m1       no                        passed

  Node Name     Running?                  Status
  ------------  ------------------------  ------------------------
  rhel6m2       no                        passed
  rhel6m1       no                        passed
Verifying Daemon "proxyt" not configured and running ...PASSED
Verifying Grid Infrastructure home path: /opt/app/12.2.0/grid ...
  Verifying '/opt/app/12.2.0/grid' ...PASSED
Verifying Grid Infrastructure home path: /opt/app/12.2.0/grid ...PASSED
Verifying User Equivalence ...PASSED
Verifying /dev/shm mounted as temporary file system ...PASSED
Verifying File system mount options for path /var ...PASSED
Verifying zeroconf check ...PASSED
Verifying ASM Filter Driver configuration ...PASSED

Pre-check for cluster services setup was unsuccessful on all the nodes.


Failures were encountered during execution of CVU verification request "stage -pre crsinst".

Verifying Swap Size ...FAILED
rhel6m2: PRVF-7573 : Sufficient swap size is not available on node "rhel6m2"
         [Required = 7.6865GB (8059836.0KB) ; Found = 4GB (4194296.0KB)]

rhel6m1: PRVF-7573 : Sufficient swap size is not available on node "rhel6m1"
         [Required = 7.6865GB (8059836.0KB) ; Found = 4GB (4194296.0KB)]


CVU operation performed:      stage -pre crsinst
Date:                         May 11, 2017 2:54:23 PM
CVU home:                     /opt/app/12.2.0/grid/
User:                         grid

6. From the GI_HOME directory run gridSetup.sh to begin the installation. Select new cluster configuration option.

12.2 has introduced many different types of clusters. This installation creates a standalone cluster.

Set cluster name and scan name. Importantly no GNS is setup or used for this installation.

Add node and node VIP names that will be used in the cluster configuration. Importantly all node roles are set as HUB nodes. If leaf nodes to be configured then this would require setting up and using GNS in previous step. Installation can proceed without a GNS if only HUB nodes are configured.

Select the network interfaces for public and ASM/Private interconnect.

The OCR and vote disks are stored in ASM but ASM could be configured either block devices or using NFS. This setup uses block devices.

As said earlier 12.2 gives the option of storing GIMR in a separate disk group. If a separate disk group is not selected then this would be stored in the same disk group as OCR and vote disks. The backups of the OCR will be stored in the disk group used for GIMR.

The first disk group created is the one for OCR and vote disk. As said earlier this expected to be of normal redundancy. Change the disk discovery path so the candidate disks are displayed. At times the disks may be visible even without changing the discovery path. Even then it's better to change the discovery path to reflect the correct path string. Otherwise during root.sh run the commands will use the default path of /dev/sd* and execution will fail.

2017/05/11 16:26:56 CLSRSC-184: Configuration of ASM failed
2017/05/11 16:27:01 CLSRSC-258: Failed to configure and start ASM
Died at /opt/app/12.2.0/grid/crs/install/crsinstall.pm line 2091.
The command '/opt/app/12.2.0/grid/perl/bin/perl -I/opt/app/12.2.0/grid/perl/lib -I/opt/app/12.2.0/grid/crs/install /opt/app/12.2.0/grid/crs/install/rootcrs.pl ' execution failed

Alert log will show that ASM disk group creation was attempted with default disk path string.

2017-05-11 16:26:24:   Invoking "/opt/app/12.2.0/grid/bin/asmca -silent -diskGroupName clusterdg 
-diskList '/dev/oracleasm/ocr3,/dev/oracleasm/ocr1,/dev/oracleasm/ocr2' -redundancy NORMAL -diskString '/dev/sd*'
-configureLocalASM -passwordFileLocation +clusterdg/orapwASM -au_size 4 " as user "grid"

If it comes to this it is possible to change the ASM_DISCOVERY_STRING in GI_HOME/crs/install/crsconfig_params to reflect the correct disk path string and run root.sh again. All of this could be avoided by explicitly setting the disk path string.

Select the disk used for OCR disk group (by default redundancy is selected as normal). Since AFD is not used in this setup, uncheck the "Configure Oracle ASM Filter Driver" option.

Select the disk used for GIMR disk group.

Select ORACLE_BASE location if not selected by default. The software location is where the software was extracted to and cannot be changed during the installation.

Running of root scripts could be automated by specify the password for root or sudo user.

The pre-requite check list warnings for ASM sharedness when block devices are used (this is not shown when AFD is used). This appear to be an issue similar to 12.1. However there's no MOS document exists for this at the time of the post. The installation was continued ignoring the warnings.

The installation setup summary (storage information shows Configure AFD : false).

When prompted run root scripts.

root.sh output for first node is shown below.

[root@rhel6m1 ~]# /opt/app/12.2.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /opt/app/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /opt/app/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /opt/app/oracle/crsdata/rhel6m1/crsconfig/rootcrs_rhel6m1_2017-05-15_12-11-53AM.log
2017/05/15 12:11:57 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2017/05/15 12:11:57 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2017/05/15 12:12:29 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2017/05/15 12:12:29 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2017/05/15 12:12:38 CLSRSC-363: User ignored prerequisites during installation
2017/05/15 12:12:38 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2017/05/15 12:12:41 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2017/05/15 12:12:44 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2017/05/15 12:12:56 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2017/05/15 12:12:59 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2017/05/15 12:12:59 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2017/05/15 12:13:37 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2017/05/15 12:13:53 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2017/05/15 12:13:53 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2017/05/15 12:14:04 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2017/05/15 12:14:19 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'
2017/05/15 12:14:49 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2017/05/15 12:14:59 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/05/15 12:15:57 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2017/05/15 12:16:07 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rhel6m1'
CRS-2676: Start of 'ora.evmd' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rhel6m1'
CRS-2676: Start of 'ora.gpnpd' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.gipcd' on 'rhel6m1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rhel6m1'
CRS-2676: Start of 'ora.diskmon' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rhel6m1' succeeded

Disk groups created successfully. Check /opt/app/oracle/cfgtoollogs/asmca/asmca-170515PM121648.log for details.


2017/05/15 12:17:45 CLSRSC-482: Running command: '/opt/app/12.2.0/grid/bin/ocrconfig -upgrade grid oinstall'
CRS-2672: Attempting to start 'ora.crf' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.storage' on 'rhel6m1'
CRS-2676: Start of 'ora.storage' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.crf' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rhel6m1'
CRS-2676: Start of 'ora.crsd' on 'rhel6m1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 5dcf6fe02fa14f27bf3597660cd83e26.
Successful addition of voting disk 4cabbda7d3694f32bff3745c4ab89112.
Successful addition of voting disk 3fb1474faee54fa2bfac45159c621cc5.
Successfully replaced voting disk group with +clusterdg.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5dcf6fe02fa14f27bf3597660cd83e26 (/dev/oracleasm/ocr3) [CLUSTERDG]
 2. ONLINE   4cabbda7d3694f32bff3745c4ab89112 (/dev/oracleasm/ocr1) [CLUSTERDG]
 3. ONLINE   3fb1474faee54fa2bfac45159c621cc5 (/dev/oracleasm/ocr2) [CLUSTERDG]
Located 3 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rhel6m1'
CRS-2677: Stop of 'ora.crsd' on 'rhel6m1' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'rhel6m1'
CRS-2673: Attempting to stop 'ora.crf' on 'rhel6m1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rhel6m1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rhel6m1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rhel6m1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rhel6m1' succeeded
CRS-2677: Stop of 'ora.crf' on 'rhel6m1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rhel6m1' succeeded
CRS-2677: Stop of 'ora.storage' on 'rhel6m1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rhel6m1'
CRS-2677: Stop of 'ora.mdnsd' on 'rhel6m1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rhel6m1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rhel6m1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rhel6m1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rhel6m1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rhel6m1'
CRS-2677: Stop of 'ora.evmd' on 'rhel6m1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rhel6m1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rhel6m1'
CRS-2677: Stop of 'ora.cssd' on 'rhel6m1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rhel6m1'
CRS-2677: Stop of 'ora.gipcd' on 'rhel6m1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2017/05/15 12:19:50 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.evmd' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rhel6m1'
CRS-2676: Start of 'ora.mdnsd' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.evmd' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rhel6m1'
CRS-2676: Start of 'ora.gpnpd' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rhel6m1'
CRS-2676: Start of 'ora.gipcd' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rhel6m1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rhel6m1'
CRS-2676: Start of 'ora.diskmon' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.ctssd' on 'rhel6m1'
CRS-2676: Start of 'ora.ctssd' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel6m1'
CRS-2676: Start of 'ora.asm' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'rhel6m1'
CRS-2676: Start of 'ora.storage' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rhel6m1'
CRS-2676: Start of 'ora.crf' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rhel6m1'
CRS-2676: Start of 'ora.crsd' on 'rhel6m1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: rhel6m1
CRS-6016: Resource auto-start has completed for server rhel6m1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/05/15 12:22:35 CLSRSC-343: Successfully started Oracle Clusterware stack
2017/05/15 12:22:35 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel6m1'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel6m1'
CRS-2676: Start of 'ora.asm' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.CLUSTERDG.dg' on 'rhel6m1'
CRS-2676: Start of 'ora.CLUSTERDG.dg' on 'rhel6m1' succeeded
2017/05/15 12:25:10 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.

2017/05/15 12:26:08 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

root.sh output for the second node.

[root@rhel6m2 ~]# /opt/app/12.2.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /opt/app/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /opt/app/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /opt/app/oracle/crsdata/rhel6m2/crsconfig/rootcrs_rhel6m2_2017-05-15_12-27-18AM.log
2017/05/15 12:27:22 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2017/05/15 12:27:22 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2017/05/15 12:27:54 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2017/05/15 12:27:54 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2017/05/15 12:27:55 CLSRSC-363: User ignored prerequisites during installation
2017/05/15 12:27:55 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2017/05/15 12:27:56 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2017/05/15 12:27:56 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2017/05/15 12:28:01 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2017/05/15 12:28:04 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2017/05/15 12:28:04 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2017/05/15 12:28:07 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2017/05/15 12:28:09 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2017/05/15 12:28:09 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2017/05/15 12:28:12 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2017/05/15 12:28:27 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'
2017/05/15 12:28:47 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2017/05/15 12:28:48 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/05/15 12:29:34 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2017/05/15 12:29:36 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rhel6m2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rhel6m2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2017/05/15 12:29:51 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rhel6m2'
CRS-2672: Attempting to start 'ora.evmd' on 'rhel6m2'
CRS-2676: Start of 'ora.mdnsd' on 'rhel6m2' succeeded
CRS-2676: Start of 'ora.evmd' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rhel6m2'
CRS-2676: Start of 'ora.gpnpd' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rhel6m2'
CRS-2676: Start of 'ora.gipcd' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rhel6m2'
CRS-2676: Start of 'ora.cssdmonitor' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rhel6m2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rhel6m2'
CRS-2676: Start of 'ora.diskmon' on 'rhel6m2' succeeded
CRS-2676: Start of 'ora.cssd' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rhel6m2'
CRS-2672: Attempting to start 'ora.ctssd' on 'rhel6m2'
CRS-2676: Start of 'ora.ctssd' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rhel6m2'
CRS-2676: Start of 'ora.crf' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rhel6m2'
CRS-2676: Start of 'ora.crsd' on 'rhel6m2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel6m2'
CRS-2676: Start of 'ora.asm' on 'rhel6m2' succeeded
CRS-6017: Processing resource auto-start for servers: rhel6m2
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.net1.network' on 'rhel6m2'
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel6m2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rhel6m1' succeeded
CRS-2676: Start of 'ora.net1.network' on 'rhel6m2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rhel6m1'
CRS-2672: Attempting to start 'ora.ons' on 'rhel6m2'
CRS-2677: Stop of 'ora.scan1.vip' on 'rhel6m1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rhel6m2'
CRS-2676: Start of 'ora.scan1.vip' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rhel6m2'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel6m2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel6m2'
CRS-2676: Start of 'ora.ons' on 'rhel6m2' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rhel6m2' succeeded
CRS-2676: Start of 'ora.asm' on 'rhel6m2' succeeded
CRS-6016: Resource auto-start has completed for server rhel6m2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/05/15 12:32:14 CLSRSC-343: Successfully started Oracle Clusterware stack
2017/05/15 12:32:14 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2017/05/15 12:32:36 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2017/05/15 12:32:49 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

At the end of the root.sh run the cluster version and mode could quried.

[root@rhel6m1 bin]#  crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [12.2.0.1.0]

[root@rhel6m1 bin]# crsctl get cluster mode status
Cluster is running in "flex" mode

[root@rhel6m1 bin]# crsctl get node role config -all
Node 'rhel6m1' configured role is 'hub'
Node 'rhel6m2' configured role is 'hub'

[root@rhel6m1 bin]#  srvctl config gns
PRKF-1110 : Neither GNS server nor GNS client is configured on this cluster

ASMCMD> showclustermode
ASM cluster : Flex mode enabled

As seen from the output above the cluster mode is flex mode. However unlike in 12.1 flex cluster, GNS was not setup during the installation. It was confirmed through a SR that setting up GNS for standalone cluster is not required even though cluster mode appear as flex.
Once the root scripts are run click OK on the configuration dialogue box to complete the creation of GIMR and other configuration assistants. At the end this clusterware installation would be completed.

With this conclude the installation of 12.2 clusterware.
As said in the beginning the GIMR will be used for storing OCR backup.

ocrconfig -showbackuploc
The Oracle Cluster Registry backup location is [+MGMTDG]

ocrconfig -showbackup auto

rhel6m1     2017/05/17 06:34:03     +MGMTDG:/ora12c2-cluster/OCRBACKUP/backup00.ocr.287.944202839     0

rhel6m1     2017/05/17 02:33:57     +MGMTDG:/ora12c2-cluster/OCRBACKUP/backup01.ocr.285.944188433     0

rhel6m1     2017/05/16 22:33:52     +MGMTDG:/ora12c2-cluster/OCRBACKUP/backup02.ocr.284.944174029     0

rhel6m1     2017/05/16 18:33:46     +MGMTDG:/ora12c2-cluster/OCRBACKUP/day.ocr.282.944159627     0

rhel6m1     2017/05/16 18:33:46     +MGMTDG:/ora12c2-cluster/OCRBACKUP/week.ocr.283.944159627     0

# ocrconfig -manualbackup

rhel6m1     2017/05/25 15:39:44     +MGMTDG:/ora12c2-cluster/OCRBACKUP/backup_20170525_153944.ocr.287.944926785     0

However, unlike in 12.1 where it gave the option of both file system and ASM for OCR backup, it seems that in 12.2 OCR backup location must be an ASM disk group.

 ocrconfig -help
Name:
        ocrconfig - Configuration tool for Oracle Cluster/Local Registry.

Synopsis:
        ocrconfig [option]
        option:
                -backuploc +<diskgroupname>         - Configure OCR backup location

Trying to set to a file system location will fail as shown below

# ocrconfig -backuploc /opt/app/12.2.0/grid/cdata/ora12c2-cluster
PROT-42: The specified backup location '/opt/app/12.2.0/grid/cdata/ora12c2-cluster' is not an ASM disk group.

At the time of writing Oracle documentation for 12.2 still state that file system is an option for OCR backups. Furthermore the clusterware admin guide states "If the OCR backup file is located in an Oracle ASM disk group, then ensure that the disk group exists and is mounted." Depending on the failure it may not be able to mount the ASM disk containing OCR backups. OCR backups could be copied out of ASM disk group using ASMCMD

asmcmd cp +MGMTDG/ora12c2-cluster/OCRBACKUP/backup_20170515_130825.ocr.281.944053705 /home/grid/
copying +MGMTDG/ora12c2-cluster/OCRBACKUP/backup_20170515_130825.ocr.281.944053705 -> /home/grid//backup_20170515_130825.ocr.281.944053705

ASM configuration list ASM instance count as 3 even though this is a 2 node cluster. This is related to flex cluster configuration where default ASM instance count is 3 even when actual node count is 2.

srvctl config asm -a
ASM home: 
Password file: +CLUSTERDG/orapwASM
Backup of Password file:
ASM listener: LISTENER
ASM is enabled.
ASM is individually enabled on nodes:
ASM is individually disabled on nodes:
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM

The ASMCA interface has also been changed in 12.2. Create the other disk group needed for data and FRA.

Using ASM Filter Driver During Installation
In order to use AFD to mange the disk sued by ASM carry out the following before the installation (before running gridSetup.sh). At this stage is assumed that grid software is extracted to grid home location. set ORACLE_HOME AND ORACLE_BASE variables as below. Oracle base is set to a temp location so that files generated during ASM disk labelling doesn't interfere with actual installation.

# export ORACLE_HOME=/opt/app/12.2.0/grid
# export ORACLE_BASE=/tmp

Run teh disk labelling command from teh $GI_HOME/bin directory.

# ./asmcmd afd_label DATA1 /dev/oracleasm/data1 --init
# ./asmcmd afd_label FRA1 /dev/oracleasm/fra1 --init

# ./asmcmd afd_label OCR1 /dev/oracleasm/ocr1 --init
# ./asmcmd afd_label OCR2 /dev/oracleasm/ocr2 --init
# ./asmcmd afd_label OCR3 /dev/oracleasm/ocr3 --init

# ./asmcmd afd_label GIMR /dev/oracleasm/gimr --init

Verify the disks are labelled.

# ./asmcmd afd_lslbl '/dev/oracleasm/*'
--------------------------------------------------------------------------------
Label                     Duplicate  Path
================================================================================
DATA1                                 /dev/oracleasm/data1
FRA1                                  /dev/oracleasm/fra1
GIMR                                  /dev/oracleasm/gimr
OCR1                                  /dev/oracleasm/ocr1
OCR2                                  /dev/oracleasm/ocr2
OCR3                                  /dev/oracleasm/ocr3

Afterwards run the gridSetup.sh as before. Give the ASM disk path string as before when it time to specify the disk for ASM. The listed disks will have the status "provisioned".

With AFD in use ASM sharedness warning is not shown as before.

The rest of the steps are the same. The summary will have Configure Oracle AFD : true.

During the root.sh run the file name for ASM diskgroup would have the AFD: prefix.

Successfully replaced voting disk group with +CLUSTERDG.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5525126aef6f4fb4bf9b383115bade73 (AFD:OCR3) [CLUSTERDG]
 2. ONLINE   a5330a1d40594f31bfcc9ee98c6b0034 (AFD:OCR2) [CLUSTERDG]
 3. ONLINE   ad0588fa57fa4f13bfd0800d4e52ea5d (AFD:OCR1) [CLUSTERDG]
Located 3 voting disk(s).

At the end of the installation process asmcmd will show following string as discovery string

asmcmd afd_dsget
AFD discovery string: /dev/oracleasm/*

while querying same from within an ASM instance will show both above disk string and AFD:*

SQL> show parameter string

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------
asm_diskstring                       string      /dev/oracleasm/*, AFD:*

v$asm_disk will have both AFD and /dev/oracleasm/* paths

SQL> select name,path from v$asm_disk order by 2;

NAME                           PATH
------------------------------ ------------------------------
                               /dev/oracleasm/data1
                               /dev/oracleasm/fra1
                               /dev/oracleasm/gimr
                               /dev/oracleasm/ocr1
                               /dev/oracleasm/ocr2
                               /dev/oracleasm/ocr3
                               AFD:DATA1
                               AFD:FRA1
GIMR                           AFD:GIMR
OCR1                           AFD:OCR1
OCR2                           AFD:OCR2
OCR3                           AFD:OCR3

However, when ASMCA is run to create additional diskgroups, the disk paths are shown as AFD paths.

Finally run the post crsisnt to verify cluster status.

$ cluvfy stage -post crsinst -n rhel6m1,rhel6m2 -verbose

Verifying Node Connectivity ...
  Verifying Hosts File ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               passed
  rhel6m2                               passed
  Verifying Hosts File ...PASSED

Interface information for node "rhel6m2"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.94    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.98    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.135   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.145   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth1   192.168.1.88    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:69:2C:B6 1500

Interface information for node "rhel6m1"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.93    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth0   192.168.0.97    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth0   192.168.0.125   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth1   192.168.1.87    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:A3:C4:6F 1500

Check: MTU consistency on the private interfaces of subnet "192.168.1.0"

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth1          192.168.1.88  192.168.1.0   1500
  rhel6m1           eth1          192.168.1.87  192.168.1.0   1500

Check: MTU consistency of the subnet "192.168.0.0".

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth0          192.168.0.94  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.98  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.135  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.145  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.93  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.97  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.125  192.168.0.0   1500

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth1:192.168.1.87]      rhel6m2[eth1:192.168.1.88]      yes

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.98]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.135]     yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.145]     yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.94]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m2[eth0:192.168.0.135]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m2[eth0:192.168.0.145]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.98]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.98]      rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.135]     rhel6m2[eth0:192.168.0.145]     yes
  rhel6m2[eth0:192.168.0.135]     rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.135]     rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.135]     rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.145]     rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.145]     rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.145]     rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.94]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.94]      rhel6m1[eth0:192.168.0.125]     yes
  rhel6m1[eth0:192.168.0.97]      rhel6m1[eth0:192.168.0.125]     yes
  Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
  Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
  Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast check ...
Checking subnet "192.168.1.0" for multicast communication with multicast group "224.0.0.251"
Verifying Multicast check ...PASSED
Verifying ASM filter driver configuration consistency ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Cluster Manager Integrity ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               running
  rhel6m2                               running
Verifying Cluster Manager Integrity ...PASSED
Verifying User Mask ...
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       0022                      0022                      passed
  rhel6m1       0022                      0022                      passed
Verifying User Mask ...PASSED
Verifying Cluster Integrity ...
  Node Name
  ------------------------------------
  rhel6m1
  rhel6m2
Verifying Cluster Integrity ...PASSED
Verifying OCR Integrity ...PASSED
Verifying CRS Integrity ...
  Verifying Clusterware Version Consistency ...PASSED
Verifying CRS Integrity ...PASSED
Verifying Node Application Existence ...

Checking existence of VIP node application (required)

  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       yes                       yes                       passed
  rhel6m1       yes                       yes                       passed


Checking existence of NETWORK node application (required)

  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       yes                       yes                       passed
  rhel6m1       yes                       yes                       passed


Checking existence of ONS node application (optional)

  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       no                        yes                       passed
  rhel6m1       no                        yes                       passed
Verifying Node Application Existence ...PASSED
Verifying Single Client Access Name (SCAN) ...
  SCAN Name         Node          Running?      ListenerName  Port          Running?
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac-scan          rhel6m1       true          LISTENER_SCAN1  1521          true
  rac-scan          rhel6m2       true          LISTENER_SCAN2  1521          true
  rac-scan          rhel6m2       true          LISTENER_SCAN3  1521          true

Checking TCP connectivity to SCAN listeners...

  Node          ListenerName              TCP connectivity?
  ------------  ------------------------  ------------------------
  rhel6m2       LISTENER_SCAN1            yes
  rhel6m2       LISTENER_SCAN2            yes
  rhel6m2       LISTENER_SCAN3            yes

  Verifying DNS/NIS name service 'rac-scan' ...
    Verifying Name Service Switch Configuration File Integrity ...PASSED

  SCAN Name     IP Address                Status                    Comment
  ------------  ------------------------  ------------------------  ----------
  rac-scan      192.168.0.145             passed
  rac-scan      192.168.0.125             passed
  rac-scan      192.168.0.135             passed
  Verifying DNS/NIS name service 'rac-scan' ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verifying OLR Integrity ...PASSED
Verifying Voting Disk ...PASSED
Verifying ASM Integrity ...
  Verifying Node Connectivity ...
    Verifying Hosts File ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m2                               passed
  rhel6m1                               passed
    Verifying Hosts File ...PASSED

Interface information for node "rhel6m2"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.94    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.98    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.135   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.145   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth1   192.168.1.88    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:69:2C:B6 1500

Interface information for node "rhel6m1"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.93    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth0   192.168.0.97    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth0   192.168.0.125   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth1   192.168.1.87    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:A3:C4:6F 1500

Check: MTU consistency on the private interfaces of subnet "192.168.1.0"

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth1          192.168.1.88  192.168.1.0   1500
  rhel6m1           eth1          192.168.1.87  192.168.1.0   1500

Check: MTU consistency of the subnet "192.168.0.0".

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth0          192.168.0.94  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.98  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.135  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.145  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.93  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.97  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.125  192.168.0.0   1500

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth1:192.168.1.87]      rhel6m2[eth1:192.168.1.88]      yes

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.98]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.135]     yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.145]     yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.94]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m2[eth0:192.168.0.135]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m2[eth0:192.168.0.145]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.98]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.98]      rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.135]     rhel6m2[eth0:192.168.0.145]     yes
  rhel6m2[eth0:192.168.0.135]     rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.135]     rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.135]     rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.145]     rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.145]     rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.145]     rhel6m1[eth0:192.168.0.125]     yes
  rhel6m2[eth0:192.168.0.94]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.94]      rhel6m1[eth0:192.168.0.125]     yes
  rhel6m1[eth0:192.168.0.97]      rhel6m1[eth0:192.168.0.125]     yes
    Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
    Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
    Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
  Verifying Node Connectivity ...PASSED
Verifying ASM Integrity ...PASSED
Verifying Device Checks for ASM ...
  Verifying Access Control List check ...
Checking consistency of device owner across all nodes...
Consistency check of device owner for "/dev/oracleasm/gimr" PASSED
Consistency check of device owner for "/dev/oracleasm/fra1" PASSED
Consistency check of device owner for "/dev/oracleasm/data1" PASSED
Consistency check of device owner for "/dev/oracleasm/ocr3" PASSED
Consistency check of device owner for "/dev/oracleasm/ocr1" PASSED
Consistency check of device owner for "/dev/oracleasm/ocr2" PASSED

Checking consistency of device group across all nodes...
Consistency check of device group for "/dev/oracleasm/gimr" PASSED
Consistency check of device group for "/dev/oracleasm/fra1" PASSED
Consistency check of device group for "/dev/oracleasm/data1" PASSED
Consistency check of device group for "/dev/oracleasm/ocr3" PASSED
Consistency check of device group for "/dev/oracleasm/ocr1" PASSED
Consistency check of device group for "/dev/oracleasm/ocr2" PASSED

Checking consistency of device permissions across all nodes...
Consistency check of device permissions for "/dev/oracleasm/gimr" PASSED
Consistency check of device permissions for "/dev/oracleasm/fra1" PASSED
Consistency check of device permissions for "/dev/oracleasm/data1" PASSED
Consistency check of device permissions for "/dev/oracleasm/ocr3" PASSED
Consistency check of device permissions for "/dev/oracleasm/ocr1" PASSED
Consistency check of device permissions for "/dev/oracleasm/ocr2" PASSED
  Verifying Access Control List check ...PASSED
Verifying Device Checks for ASM ...PASSED
Verifying ASM disk group free space ...PASSED
Verifying I/O scheduler ...
  Verifying Package: cvuqdisk-1.0.10-1 ...PASSED
Verifying I/O scheduler ...PASSED
Verifying User Not In Group "root": grid ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    does not exist
  rhel6m1       passed                    does not exist
Verifying User Not In Group "root": grid ...PASSED
Verifying Clock Synchronization ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               passed
  rhel6m2                               passed

  Node Name                             State
  ------------------------------------  ------------------------
  rhel6m2                               Active
  rhel6m1                               Active

  Node Name     Time Offset               Status
  ------------  ------------------------  ------------------------
  rhel6m2       0.0                       passed
  rhel6m1       0.0                       passed
Verifying Clock Synchronization ...PASSED
Verifying VIP Subnet configuration check ...PASSED
Verifying Network configuration consistency checks ...PASSED
Verifying File system mount options for path GI_HOME ...PASSED

Post-check for cluster services setup was successful.

CVU operation performed:      stage -post crsinst
Date:                         May 15, 2017 1:18:38 PM
CVU home:                     /opt/app/12.2.0/grid/
User:                         grid

The next step is to install the database software and create the database.

Related Posts
Installing 12c (12.1.0.2) Flex Cluster on RHEL 6 with Role Separation
Installing 12c (12.1.0.1) RAC on RHEL 6 with Role Separation - Clusterware
Installing 11gR2 (11.2.0.3) GI with Role Separation on RHEL 6
Installing 11gR2 (11.2.0.3) GI with Role Separation on OEL 6
Installing 11gR2 Standalone Server with ASM and Role Separation on RHEL 6
11gR2 Standalone Data Guard (with ASM and Role Separation)

↧

Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Database

June 1, 2017, 12:00 am

≫ Next: Boot in Single User Mode on RHEL 7

≪ Previous: Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Clusterware

Following the clusterware installation the next step is to install database software and create the database. The database software isn't much different from that of 12.1 database software installation. Even though the oracle installation guides states gcc and gcc-c++ are not required for RHEL, the cluvfy still searches for them.

$ cluvfy stage -pre dbinst -n rhel6m1,rhel6m2 -r 12.2 -verbose

Verifying Physical Memory ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       7.6865GB (8059836.0KB)    1GB (1048576.0KB)         passed
  rhel6m1       7.6865GB (8059836.0KB)    1GB (1048576.0KB)         passed
Verifying Physical Memory ...PASSED
Verifying Available Physical Memory ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       6.1648GB (6464272.0KB)    50MB (51200.0KB)          passed
  rhel6m1       5.0643GB (5310272.0KB)    50MB (51200.0KB)          passed
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       4GB (4194296.0KB)         7.6865GB (8059836.0KB)    failed
  rhel6m1       4GB (4194296.0KB)         7.6865GB (8059836.0KB)    failed
Verifying Swap Size ...FAILED (PRVF-7573)
Verifying Free Space: rhel6m2:/tmp ...
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /tmp              rhel6m2       /             23.0566GB     1GB           passed
Verifying Free Space: rhel6m2:/tmp ...PASSED
Verifying Free Space: rhel6m1:/tmp ...
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /tmp              rhel6m1       /             19.7646GB     1GB           passed
Verifying Free Space: rhel6m1:/tmp ...PASSED
Verifying User Existence: grid ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists(502)
  rhel6m1       passed                    exists(502)

  Verifying Users With Same UID: 502 ...PASSED
Verifying User Existence: grid ...PASSED
Verifying Group Existence: dba ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists
  rhel6m1       passed                    exists
Verifying Group Existence: dba ...PASSED
Verifying Group Existence: oinstall ...
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rhel6m2       passed                    exists
  rhel6m1       passed                    exists
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: dba ...
  Node Name         User Exists   Group Exists  User in Group  Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           yes           yes           yes           passed
  rhel6m1           yes           yes           yes           passed
Verifying Group Membership: dba ...PASSED
Verifying Group Membership: oinstall(Primary) ...
  Node Name         User Exists   Group Exists  User in Group  Primary       Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m2           yes           yes           yes           yes           passed
  rhel6m1           yes           yes           yes           yes           passed
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Run Level ...
  Node Name     run level                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       3                         3,5                       passed
  rhel6m1       3                         3,5                       passed
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           hard          65536         65536         passed
  rhel6m1           hard          65536         65536         passed
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           soft          1024          1024          passed
  rhel6m1           soft          1024          1024          passed
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           hard          16384         16384         passed
  rhel6m1           hard          16384         16384         passed
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           soft          2047          2047          passed
  rhel6m1           soft          2047          2047          passed
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           soft          10240         10240         passed
  rhel6m1           soft          10240         10240         passed
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Architecture ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       x86_64                    x86_64                    passed
  rhel6m1       x86_64                    x86_64                    passed
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       2.6.32-358.el6.x86_64     2.6.32                    passed
  rhel6m1       2.6.32-358.el6.x86_64     2.6.32                    passed
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           3010          3010          250           passed
  rhel6m2           3010          3010          250           passed
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           385280        385280        32000         passed
  rhel6m2           385280        385280        32000         passed
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           3010          3010          100           passed
  rhel6m2           3010          3010          100           passed
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           128           128           128           passed
  rhel6m2           128           128           128           passed
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           68719476736   68719476736   4126636032    passed
  rhel6m2           68719476736   68719476736   4126636032    passed
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4096          4096          4096          passed
  rhel6m2           4096          4096          4096          passed
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4294967296    4294967296    805983        passed
  rhel6m2           4294967296    4294967296    805983        passed
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           6815744       6815744       6815744       passed
  rhel6m2           6815744       6815744       6815744       passed
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           between 9000 & 65500  between 9000 & 65500  between 9000 & 65535  passed
  rhel6m2           between 9000 & 65500  between 9000 & 65500  between 9000 & 65535  passed
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4194304       4194304       262144        passed
  rhel6m2           4194304       4194304       262144        passed
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           4194304       4194304       4194304       passed
  rhel6m2           4194304       4194304       4194304       passed
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           1048576       1048576       262144        passed
  rhel6m2           1048576       1048576       262144        passed
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           2097152       2097152       1048576       passed
  rhel6m2           2097152       2097152       1048576       passed
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rhel6m1           3145728       3145728       1048576       passed
  rhel6m2           3145728       3145728       1048576       passed
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying Package: binutils-2.20.51.0.2 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
  rhel6m1       binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
Verifying Package: binutils-2.20.51.0.2 ...PASSED
Verifying Package: compat-libcap1-1.10 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       compat-libcap1-1.10-1     compat-libcap1-1.10       passed
  rhel6m1       compat-libcap1-1.10-1     compat-libcap1-1.10       passed
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
  rhel6m1       compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...PASSED
Verifying Package: libgcc-4.4.7 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libgcc(x86_64)-4.4.7-3.el6  libgcc(x86_64)-4.4.7      passed
  rhel6m1       libgcc(x86_64)-4.4.7-3.el6  libgcc(x86_64)-4.4.7      passed
Verifying Package: libgcc-4.4.7 (x86_64) ...PASSED
Verifying Package: libstdc++-4.4.7 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libstdc++(x86_64)-4.4.7-3.el6  libstdc++(x86_64)-4.4.7   passed
  rhel6m1       libstdc++(x86_64)-4.4.7-3.el6  libstdc++(x86_64)-4.4.7   passed
Verifying Package: libstdc++-4.4.7 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.4.7 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libstdc++-devel(x86_64)-4.4.7-3.el6  libstdc++-devel(x86_64)-4.4.7  passed
  rhel6m1       libstdc++-devel(x86_64)-4.4.7-3.el6  libstdc++-devel(x86_64)-4.4.7  passed
Verifying Package: libstdc++-devel-4.4.7 (x86_64) ...PASSED
Verifying Package: sysstat-9.0.4 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
  rhel6m1       sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
Verifying Package: sysstat-9.0.4 ...PASSED
Verifying Package: gcc-4.4.7 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       gcc-4.4.7-3.el6           gcc-4.4.7                 passed
  rhel6m1       gcc-4.4.7-3.el6           gcc-4.4.7                 passed
Verifying Package: gcc-4.4.7 ...PASSED
Verifying Package: gcc-c++-4.4.7 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       gcc-c++-4.4.7-3.el6       gcc-c++-4.4.7             passed
  rhel6m1       gcc-c++-4.4.7-3.el6       gcc-c++-4.4.7             passed
Verifying Package: gcc-c++-4.4.7 ...PASSED
Verifying Package: ksh ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       ksh                       ksh                       passed
  rhel6m1       ksh                       ksh                       passed
Verifying Package: ksh ...PASSED
Verifying Package: make-3.81 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       make-3.81-20.el6          make-3.81                 passed
  rhel6m1       make-3.81-20.el6          make-3.81                 passed
Verifying Package: make-3.81 ...PASSED
Verifying Package: glibc-2.12 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       glibc(x86_64)-2.12-1.107.el6  glibc(x86_64)-2.12        passed
  rhel6m1       glibc(x86_64)-2.12-1.107.el6  glibc(x86_64)-2.12        passed
Verifying Package: glibc-2.12 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.12 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       glibc-devel(x86_64)-2.12-1.107.el6  glibc-devel(x86_64)-2.12  passed
  rhel6m1       glibc-devel(x86_64)-2.12-1.107.el6  glibc-devel(x86_64)-2.12  passed
Verifying Package: glibc-devel-2.12 (x86_64) ...PASSED
Verifying Package: libaio-0.3.107 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
  rhel6m1       libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
Verifying Package: libaio-0.3.107 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.107 (x86_64) ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
  rhel6m1       libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
Verifying Package: libaio-devel-0.3.107 (x86_64) ...PASSED
Verifying Package: smartmontools-5.43-1 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       smartmontools-5.43-1.el6  smartmontools-5.43-1      passed
  rhel6m1       smartmontools-5.43-1.el6  smartmontools-5.43-1      passed
Verifying Package: smartmontools-5.43-1 ...PASSED
Verifying Package: net-tools-1.60-110 ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       net-tools-1.60-110.el6_2  net-tools-1.60-110        passed
  rhel6m1       net-tools-1.60-110.el6_2  net-tools-1.60-110        passed
Verifying Package: net-tools-1.60-110 ...PASSED
Verifying Users With Same UID: 0 ...PASSED
Verifying Current Group ID ...PASSED
Verifying Root user consistency ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m2                               passed
  rhel6m1                               passed
Verifying Root user consistency ...PASSED
Verifying Node Connectivity ...
  Verifying Hosts File ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               passed
  rhel6m2                               passed
  Verifying Hosts File ...PASSED

Interface information for node "rhel6m2"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.94    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.125   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth0   192.168.0.98    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:7E:61:A9 1500
 eth1   192.168.1.88    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:69:2C:B6 1500

Interface information for node "rhel6m1"

 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   192.168.0.93    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth0   192.168.0.97    192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth0   192.168.0.145   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth0   192.168.0.135   192.168.0.0     0.0.0.0         192.168.0.100   08:00:27:AC:F3:CC 1500
 eth1   192.168.1.87    192.168.1.0     0.0.0.0         192.168.0.100   08:00:27:A3:C4:6F 1500

Check: MTU consistency on the private interfaces of subnet "192.168.1.0"

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth1          192.168.1.88  192.168.1.0   1500
  rhel6m1           eth1          192.168.1.87  192.168.1.0   1500

Check: MTU consistency of the subnet "192.168.0.0".

  Node              Name          IP Address    Subnet        MTU
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           eth0          192.168.0.94  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.125  192.168.0.0   1500
  rhel6m2           eth0          192.168.0.98  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.93  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.97  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.145  192.168.0.0   1500
  rhel6m1           eth0          192.168.0.135  192.168.0.0   1500

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth1:192.168.1.87]      rhel6m2[eth1:192.168.1.88]      yes

  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.125]     yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.98]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m2[eth0:192.168.0.94]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m1[eth0:192.168.0.93]      rhel6m1[eth0:192.168.0.145]     yes
  rhel6m1[eth0:192.168.0.93]      rhel6m1[eth0:192.168.0.135]     yes
  rhel6m2[eth0:192.168.0.125]     rhel6m2[eth0:192.168.0.98]      yes
  rhel6m2[eth0:192.168.0.125]     rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.125]     rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.125]     rhel6m1[eth0:192.168.0.145]     yes
  rhel6m2[eth0:192.168.0.125]     rhel6m1[eth0:192.168.0.135]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m2[eth0:192.168.0.94]      yes
  rhel6m2[eth0:192.168.0.98]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.98]      rhel6m1[eth0:192.168.0.145]     yes
  rhel6m2[eth0:192.168.0.98]      rhel6m1[eth0:192.168.0.135]     yes
  rhel6m2[eth0:192.168.0.94]      rhel6m1[eth0:192.168.0.97]      yes
  rhel6m2[eth0:192.168.0.94]      rhel6m1[eth0:192.168.0.145]     yes
  rhel6m2[eth0:192.168.0.94]      rhel6m1[eth0:192.168.0.135]     yes
  rhel6m1[eth0:192.168.0.97]      rhel6m1[eth0:192.168.0.145]     yes
  rhel6m1[eth0:192.168.0.97]      rhel6m1[eth0:192.168.0.135]     yes
  rhel6m1[eth0:192.168.0.145]     rhel6m1[eth0:192.168.0.135]     yes
  Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
  Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
  Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast check ...
Checking subnet "192.168.1.0" for multicast communication with multicast group "224.0.0.251"
Verifying Multicast check ...PASSED
Verifying User Mask ...
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       0022                      0022                      passed
  rhel6m1       0022                      0022                      passed
Verifying User Mask ...PASSED
Verifying CRS Integrity ...
  Verifying Clusterware Version Consistency ...PASSED
Verifying CRS Integrity ...PASSED
Verifying Cluster Manager Integrity ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               running
  rhel6m2                               running
Verifying Cluster Manager Integrity ...PASSED
Verifying Node Application Existence ...

Checking existence of VIP node application (required)

  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       yes                       yes                       passed
  rhel6m1       yes                       yes                       passed


Checking existence of NETWORK node application (required)

  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       yes                       yes                       passed
  rhel6m1       yes                       yes                       passed


Checking existence of ONS node application (optional)

  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       no                        yes                       passed
  rhel6m1       no                        yes                       passed
Verifying Node Application Existence ...PASSED
Verifying Clock Synchronization ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m1                               passed
  rhel6m2                               passed

  Node Name                             State
  ------------------------------------  ------------------------
  rhel6m2                               Active
  rhel6m1                               Active

  Node Name     Time Offset               Status
  ------------  ------------------------  ------------------------
  rhel6m2       0.0                       passed
  rhel6m1       0.0                       passed
Verifying Clock Synchronization ...PASSED
Verifying resolv.conf Integrity ...
  Verifying (Linux) resolv.conf Integrity ...
  Node Name                             Status
  ------------------------------------  ------------------------
  rhel6m2                               passed
  rhel6m1                               passed

  checking response for name "rhel6m2" from each of the name servers specified
  in "/etc/resolv.conf"

  Node Name     Source                    Comment                   Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m2       192.168.0.66              IPv4                      passed

  checking response for name "rhel6m1" from each of the name servers specified
  in "/etc/resolv.conf"

  Node Name     Source                    Comment                   Status
  ------------  ------------------------  ------------------------  ----------
  rhel6m1       192.168.0.66              IPv4                      passed
  Verifying (Linux) resolv.conf Integrity ...PASSED
Verifying resolv.conf Integrity ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Single Client Access Name (SCAN) ...
  SCAN Name         Node          Running?      ListenerName  Port          Running?
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac-scan          rhel6m2       true          LISTENER_SCAN1  1521          true
  rac-scan          rhel6m1       true          LISTENER_SCAN2  1521          true
  rac-scan          rhel6m1       true          LISTENER_SCAN3  1521          true

Checking TCP connectivity to SCAN listeners...

  Node          ListenerName              TCP connectivity?
  ------------  ------------------------  ------------------------
  rhel6m2       LISTENER_SCAN1            yes
  rhel6m2       LISTENER_SCAN2            yes
  rhel6m2       LISTENER_SCAN3            yes

  Verifying DNS/NIS name service 'rac-scan' ...
    Verifying Name Service Switch Configuration File Integrity ...PASSED

  SCAN Name     IP Address                Status                    Comment
  ------------  ------------------------  ------------------------  ----------
  rac-scan      192.168.0.145             passed
  rac-scan      192.168.0.125             passed
  rac-scan      192.168.0.135             passed
  Verifying DNS/NIS name service 'rac-scan' ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verifying VIP Subnet configuration check ...PASSED
Verifying Database Clusterware Version Compatibility ...PASSED
Verifying ASM storage privileges for the user: grid ...
  Verifying Group Membership: asmdba ...
  Node Name         User Exists   Group Exists  User in Group  Status
  ----------------  ------------  ------------  ------------  ----------------
  rhel6m2           yes           yes           yes           passed
  rhel6m1           yes           yes           yes           passed
  Verifying Group Membership: asmdba ...PASSED
Verifying ASM storage privileges for the user: grid ...PASSED
Verifying Daemon "proxyt" not configured and running ...
  Node Name     Configured                Status
  ------------  ------------------------  ------------------------
  rhel6m2       no                        passed
  rhel6m1       no                        passed

  Node Name     Running?                  Status
  ------------  ------------------------  ------------------------
  rhel6m2       no                        passed
  rhel6m1       no                        passed
Verifying Daemon "proxyt" not configured and running ...PASSED
Verifying ACFS device special file ...PASSED
Verifying /dev/shm mounted as temporary file system ...PASSED
Verifying Maximum locked memory check ...FAILED (PRVE-0059)

Pre-check for database installation was unsuccessful on all the nodes.


Failures were encountered during execution of CVU verification request "stage -pre dbinst".

Verifying Swap Size ...FAILED
rhel6m2: PRVF-7573 : Sufficient swap size is not available on node "rhel6m2"
         [Required = 7.6865GB (8059836.0KB) ; Found = 4GB (4194296.0KB)]

rhel6m1: PRVF-7573 : Sufficient swap size is not available on node "rhel6m1"
         [Required = 7.6865GB (8059836.0KB) ; Found = 4GB (4194296.0KB)]

Verifying Maximum locked memory check ...FAILED
rhel6m2: PRVE-0059 : no default entry or entry specific to user "grid" was
         found in the configuration file "/etc/security/limits.conf" when
         checking the maximum locked memory limit on node "rhel6m2.domain.net"

rhel6m1: PRVE-0059 : no default entry or entry specific to user "grid" was
         found in the configuration file "/etc/security/limits.conf" when
         checking the maximum locked memory limit on node "rhel6m1.domain.net"


CVU operation performed:      stage -pre dbinst
Date:                         May 17, 2017 12:36:35 PM
CVU home:                     /opt/app/12.2.0/grid/
User:                         grid

Execute runInstaller to commence the database software installation.

As mentioned earlier a new user group for RAC management has been introduced in 12.2. This could be specified as the RAC administrative group when specifying privilege OS groups.

Summary

Once the database software is installed next step is to create the database. Before running DBCA make sure oracle user has write permission on $ORACLE_BASE/cfgtoollogs, $ORACLE_BASE/cfgtoollogs/dbca and $ORACLE_BASE/cfgtoollogs/sqlpatch directories on the local node. Cluvfy could be used to check pre-requisite for RAC DB (output is without verbose option).

cluvfy stage -pre dbcfg -n rhel6m1,rhel6m2 -d /opt/app/oracle/product/12.2.0/dbhome_1

Verifying Physical Memory ...PASSED
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...FAILED (PRVF-7573)
Verifying Free Space: rhel6m2:/tmp ...PASSED
Verifying Free Space: rhel6m1:/tmp ...PASSED
Verifying User Existence: oracle ...
  Verifying Users With Same UID: 501 ...PASSED
Verifying User Existence: oracle ...PASSED
Verifying Group Existence: dgdba ...PASSED
Verifying Group Existence: dba ...PASSED
Verifying Group Existence: racdba ...PASSED
Verifying Group Existence: backupdba ...PASSED
Verifying Group Existence: oper ...PASSED
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: backupdba ...PASSED
Verifying Group Membership: racdba ...PASSED
Verifying Group Membership: dgdba ...PASSED
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Group Membership: dba ...PASSED
Verifying Group Membership: oper ...PASSED
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying Package: binutils-2.20.51.0.2 ...PASSED
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...PASSED
Verifying Package: libgcc-4.4.7 (x86_64) ...PASSED
Verifying Package: libstdc++-4.4.7 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.4.7 (x86_64) ...PASSED
Verifying Package: sysstat-9.0.4 ...PASSED
Verifying Package: gcc-4.4.7 ...PASSED
Verifying Package: gcc-c++-4.4.7 ...PASSED
Verifying Package: ksh ...PASSED
Verifying Package: make-3.81 ...PASSED
Verifying Package: glibc-2.12 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.12 (x86_64) ...PASSED
Verifying Package: libaio-0.3.107 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.107 (x86_64) ...PASSED
Verifying Package: smartmontools-5.43-1 ...PASSED
Verifying Package: net-tools-1.60-110 ...PASSED
Verifying Current Group ID ...PASSED
Verifying CRS Integrity ...
  Verifying Clusterware Version Consistency ...PASSED
Verifying CRS Integrity ...PASSED
Verifying Node Application Existence ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Single Client Access Name (SCAN) ...
  Verifying DNS/NIS name service 'rac-scan' ...
    Verifying Name Service Switch Configuration File Integrity ...PASSED
  Verifying DNS/NIS name service 'rac-scan' ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verifying ASM Integrity ...
  Verifying Node Connectivity ...
    Verifying Hosts File ...PASSED
    Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
    Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
    Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
  Verifying Node Connectivity ...PASSED
Verifying ASM Integrity ...PASSED
Verifying Database Clusterware Version Compatibility ...PASSED
Verifying File system mount options for path ORACLE_HOME ...PASSED
Verifying /dev/shm mounted as temporary file system ...PASSED

Pre-check for database configuration was unsuccessful on all the nodes.

Failures were encountered during execution of CVU verification request "stage -pre dbcfg".

Verifying Swap Size ...FAILED

As this is a test system, the failure due to insufficient SWAP size is ignorable. The DBCA interfaces are different on 12.2 compared to earlier version of 12.1.

The database created here is a non-CDB. Though this is depreciated, it it still possible to create non-CDB on 12.2.

Select ASM for storage

Select / de-select DB options as required.

Set configuration options. On 12.2 the database character set defaults to AL2UTF8.

Customize Storage, including control file options.

Create the database, generate template or scripts.

DBCA runs a pre-requisite check.

Summary

Completion

This concludes the creation of non-CDB on 12.2. One notable difference on 12.2 DB creation compared to 12.1 is when creating CDBs. Unlike in 12.1, in the 12.2 DBCA it is allowed to select subset of DB options during the database creation. In 12.1 this had to be done using scripts (Refer 2001512.1 and 1616554.1).
The database is created as CDB.

DBCA allows selecting subset of options similar to non-CDB.

Summary page showing database is container DB.

Create the pluggable database

Querying the registry shows the subset of options being installed.

SQL> show con_name

CON_NAME
------------------------------
PDB12C2
SQL> select comp_id,comp_name,status,version from dba_registry;

COMP_ID    COMP_NAME                                STATUS   VERSION
---------- ---------------------------------------- -------- ---------------
CATALOG    Oracle Database Catalog Views            VALID    12.2.0.1.0
CATPROC    Oracle Database Packages and Types       VALID    12.2.0.1.0
RAC        Oracle Real Application Clusters         VALID    12.2.0.1.0
XDB        Oracle XML Database                      VALID    12.2.0.1.0
OWM        Oracle Workspace Manager                 VALID    12.2.0.1.0

As mentioned in the previous post the standalone cluster mode is flex. This allows database instance to function even when the ASM instance on the node fails. This could be tested using the same steps listed in converting standard cluster to flex cluster post. Following is the status before ASM is shutdown on node 2. This shows ASM instance up on both instances and DB as a client.

[grid@rhel6m1 ~]$ srvctl status asm -detail
ASM is running on rhel6m2,rhel6m1
ASM is enabled.
ASM instance +ASM1 is running on node rhel6m1
Number of connected clients: 3
Client names: -MGMTDB:_mgmtdb:ora12c2-cluster cdb12c21:cdb12c2:ora12c2-cluster rhel6m1.domain.net:_OCR:ora12c2-cluster
ASM instance +ASM2 is running on node rhel6m2
Number of connected clients: 2
Client names: cdb12c22:cdb12c2:ora12c2-cluster rhel6m2.domain.net:_OCR:ora12c2-cluster

Stop the ASM on node 2 with

[grid@rhel6m1 ~]$ srvctl stop asm -node rhel6m2 -force

The database instance on node two would be running by becoming a client of ASM1.

[grid@rhel6m1 ~]$ srvctl status database -d cdb12c2
Instance cdb12c21 is running on node rhel6m1
Instance cdb12c22 is running on node rhel6m2

[grid@rhel6m1 ~]$ srvctl status asm -detail
ASM is running on rhel6m1
ASM is enabled.
ASM instance +ASM1 is running on node rhel6m1
Number of connected clients: 4
Client names: -MGMTDB:_mgmtdb:ora12c2-cluster cdb12c21:cdb12c2:ora12c2-cluster cdb12c22:cdb12c2:ora12c2-cluster rhel6m1.domain.net:_OCR:ora12c2-cluster

Starting the ASM instance on node 2 again will relocate the database instance to local ASM instance.

[grid@rhel6m2 ~]$ srvctl start asm -node rhel6m2

[grid@rhel6m1 ~]$ srvctl status asm -detail
ASM is running on rhel6m2,rhel6m1
ASM is enabled.
ASM instance +ASM1 is running on node rhel6m1
Number of connected clients: 3
Client names: -MGMTDB:_mgmtdb:ora12c2-cluster cdb12c21:cdb12c2:ora12c2-cluster rhel6m1.domain.net:_OCR:ora12c2-cluster
ASM instance +ASM2 is running on node rhel6m2
Number of connected clients: 2
Client names: cdb12c22:cdb12c2:ora12c2-cluster rhel6m2.domain.net:_OCR:ora12c2-cluster

Unlike in 12.1 where relocation had to be done manually, on 12.2 the relocation process happens automatically. Following could be observed on alert logs of node 1 and node 2. From node 1 ASM alert log

2017-05-31T10:13:05.648771+01:00
NOTE: relocating client cdb12c22:cdb12c2:ora12c2-cluster (reg:2404143953) to its local instance; Message from node 2 - ASM Instance +ASM2 is up
NOTE: cleaned up ASM client cdb12c22:cdb12c2:ora12c2-cluster connection state (reg:2404143953)

From node 2 ASM alert log

NOTE: registered owner id 0x10006 for cdb12c22:cdb12c2:ora12c2-cluster (reconnecting)
NOTE: Flex client cdb12c22:cdb12c2:ora12c2-cluster re-registered, osid 22652, mbr 0x0, asmb 28041 (reg:163318476)

↧

Boot in Single User Mode on RHEL 7

June 11, 2017, 12:00 am

≫ Next: Enabling Automatic AWR Snapshots on PDB

≪ Previous: Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Database

On RHEl7 booting in single user mode is done with the use of rescue mode. RHEL7 documentation states that in RHEL7 rescue mode is equivalent to single user mode. In this mode the system attempts to mount all local file systems and start some important system services. However, it does not activate network interfaces or allow more users to login to the system at the same time.
To boot in the single user mode (rescue mode) press e (edit) on the GRUB2 boot screen.

Scroll down to the line beginning with "linux16" (on x86_64 bit systems) For other systems refer RHEL 7 documentation. On the following screenshot this is underlined in red. Add the following to the end of this line (underline in green in the image below).

systemd.unit=rescue.target

Press Ctrl+x to boot the system in the rescue mode. Enter the root password when prompted to gain access to the system in single user mode.

↧

Enabling Automatic AWR Snapshots on PDB

June 17, 2017, 12:00 am

≫ Next: Upgrading to 12.2.0.1 Using 12.1.0.2 Backups

≪ Previous: Boot in Single User Mode on RHEL 7

Oracle 12.2 allows AWR snapshots to be taken both at CDB and PDB level. CDB level snapshots will contain both CDB and PDB statistics.

AWR report will list the containers whose statistics are available in the AWR report. Below screenshot shows part of SQL statistics section of a CDB level AWR report. It could be seen it has SQL stats for both PDB and the root CDB.

On the other-hand PDB level snapshots will have statistics local to that PDB and some global statistics. Read here for further information on these.

Automatic snapshot is only enabled on the CDB level and by default is disabled on PDB. Oracle recommendation is "to generally use manual snapshots for a PDB. You should enable automatic snapshots only selectively for a PDB for performance reasons".
However, if required automatic AWR snapshots could be enabled on PDB by setting awr_pdb_autoflush_enabled to true. This could be set on CDB root level or PDB level. If set to true on CDB root level then automatic snapshots will be enabled for all the PDBs. If set for PDB then automatic AWR snapshot is enabled only for that PDB. Once awr_pdb_autoflush_enabled is set to true login to the PDB and run DBMS_WORKLOAD_REPOSITORY.MODIFY_SNAPSHOT_SETTINGS with an interval value greater than 0.
Below steps shows enable automatic AWR snapshots for all the PDBs in a CDB.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 APDB                           READ WRITE NO

Enable awr_pdb_autoflush to true on cdb

SQL> show con_name

CON_NAME
---------
CDB$ROOT

SQL>  alter system set awr_pdb_autoflush_enabled=true scope=both sid='*';
System altered.

SQL> show con_name;

CON_NAME
---------
APDB

SQL> select * from awr_pdb_wr_control;

      DBID SNAP_INTERVAL        RETENTION            TOPNSQL        CON_ID
---------- -------------------- -------------------- ---------- ----------
3913427709 +40150 00:01:00.0    +00008 00:00:00.0    DEFAULT             3


BEGIN
 DBMS_WORKLOAD_REPOSITORY.MODIFY_SNAPSHOT_SETTINGS( retention => 11520, 
                                                           interval => 60,
                                                           topnsql => 34,
                                                           dbid =>3913427709);
END;
/

SQL> select * from awr_pdb_wr_control;

      DBID SNAP_INTERVAL        RETENTION            TOPNSQL        CON_ID
---------- -------------------- -------------------- ---------- ----------
3913427709 +00000 01:00:00.0    +00008 00:00:00.0            34          3

AWR_PDB_WR_CONTROL view shows that interval has been set to every one hour.

When the first snapshot is generated, the AWR_PDB_DATABASE_INSTANCE view will be populated with PDB data.

SQL> select con_id,dbid,INSTANCE_NUMBER,INSTANCE_NAME,CDB,LAST_ASH_SAMPLE_ID from AWR_PDB_DATABASE_INSTANCE;

    CON_ID       DBID INSTANCE_NUMBER INSTANCE_NAME    CDB LAST_ASH_SAMPLE_ID
---------- ---------- --------------- ---------------- --- ------------------
         3 3913427709               2 cdb12c22         YES            1102638
         3 3913427709               1 cdb12c21         YES            1105261

Running awrrpt.sql before a record for PDB exists in AWR_PDB_DATABASE_INSTANCE view will result in "ORA-20200: Database/Instance 3913427709/1 does not exist in AWR_PDB_DATABASE_INSTANCE" (refer 2267849.1). When running PDB AWR reports use AWR_PDB as the location for the AWR_DATA.

Specify the location of AWR Data
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
AWR_ROOT - Use AWR data from root (default)
AWR_PDB - Use AWR data from PDB
Enter value for awr_location: AWR_PDB

There is another optional parameter to consider when enable automatic AWR snapshots for PDBs. AWR_SNAPSHOT_TIME_OFFSET specifies an off-set for snapshot start time. When there are large number of PDBs in a CDB, setting same snapshot time for all of them could result in CPU spikes. Setting this to special value of 1000000 will result in an off-set based on the DB name, reducing CPU spikes due to multiple PDBs taking snapshot at the same time.

SQL> alter system set awr_snapshot_time_offset=1000000 scope=both sid='*';
System altered.

In summary to enable automatic snapshot taking on PDB, set awr_pdb_autoflush_enabled to true and set snapshot interval to value greater than 0.

Useful metalink notes
ORA-20200 Error When Generating AWR or ADDM Report as a PDB DBA User From a 12.2.0.1 CDB Database [ID 2267849.1]

↧

Upgrading to 12.2.0.1 Using 12.1.0.2 Backups

July 1, 2017, 12:00 am

≫ Next: Upgrading to 12.2.0.1 Using 11.2.0.4 Backups

≪ Previous: Enabling Automatic AWR Snapshots on PDB

This post list the steps for upgrading a 12.1.0.2 single instance CDB to 12.2.0.1 using the backups of 12.1.0.2 DB. The database is restored using the higher version (12.2.0.1) oracle binaries. The steps below show the restore and upgrade process for SE2 CDB with one PDB. It is assumed that a backup of the database is taken and copied to server where 12.2.0.1 software is installed. A previous post is available which shows upgrading from 11.2.0.4 to 12.1 using backups.
1. Set the Oracle home to 12cR2 and oracle SID to the database to be restored.

 echo $ORACLE_HOME
/opt/app/oracle/product/12.2.0/sehome_1

export ORACLE_SID=std12cdb

2. Restore and recover the database

RMAN> startup nomount;
RMAN> create spfile from pfile;
RMAN> startup force nomount;

RMAN> restore controlfile from '/home/oracle/backup/control.bkp';
Starting restore at 08-JUN-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=1 device type=DISK

channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
output file name=/opt/app/oracle/oradata/STD12CDB/controlfile/o1_mf_dmx12r7y_.ctl
output file name=/opt/app/oracle/fast_recovery/STD12CDB/controlfile/o1_mf_dmx12rk3_.ctl
Finished restore at 08-JUN-17

RMAN> alter database mount;
Statement processed
released channel: ORA_DISK_1

RMAN> crosscheck backupset;
Starting implicit crosscheck backup at 08-JUN-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=301 device type=DISK
Crosschecked 5 objects
Finished implicit crosscheck backup at 08-JUN-17

Starting implicit crosscheck copy at 08-JUN-17
using channel ORA_DISK_1
Finished implicit crosscheck copy at 08-JUN-17

searching for all files in the recovery area
cataloging files...
no files cataloged

using channel ORA_DISK_1
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/usr/local/backups/db02s675bb_1_1.bkp RECID=2 STAMP=946050411
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/usr/local/backups/db07s696ij_1_1.bkp RECID=7 STAMP=946117203
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/usr/local/backups/db08s696iq_1_1.bkp RECID=8 STAMP=946117210
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/usr/local/backups/db09s696it_1_1.bkp RECID=9 STAMP=946117213
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/ssdoracle/flash_recovery/STD12CDB/autobackup/2017_06_08/o1_mf_s_946116946_dml5pk8q_.bkp RECID=10 STAMP=946117217
Crosschecked 5 objects


RMAN> delete noprompt expired backup;
using channel ORA_DISK_1
deleted backup piece
backup piece handle=/usr/local/backups/db02s675bb_1_1.bkp RECID=2 STAMP=946050411
deleted backup piece
backup piece handle=/usr/local/backups/db07s696ij_1_1.bkp RECID=7 STAMP=946117203
deleted backup piece
backup piece handle=/usr/local/backups/db08s696iq_1_1.bkp RECID=8 STAMP=946117210
deleted backup piece
backup piece handle=/usr/local/backups/db09s696it_1_1.bkp RECID=9 STAMP=946117213
deleted backup piece
backup piece handle=/ssdoracle/flash_recovery/STD12CDB/autobackup/2017_06_08/o1_mf_s_946116946_dml5pk8q_.bkp RECID=10 STAMP=946117217
Deleted 5 EXPIRED objects


RMAN> catalog  start with '/home/oracle/backup/' noprompt;
searching for all files that match the pattern /home/oracle/backup/

List of Files Unknown to the Database
=====================================
File Name: /home/oracle/backup/res.rmn
File Name: /home/oracle/backup/pfile.ora
File Name: /home/oracle/backup/control.bkp
File Name: /home/oracle/backup/db07s696ij_1_1.bkp
File Name: /home/oracle/backup/db08s696iq_1_1.bkp
File Name: /home/oracle/backup/db09s696it_1_1.bkp
cataloging files...
cataloging done

List of Cataloged Files
=======================
File Name: /home/oracle/backup/control.bkp
File Name: /home/oracle/backup/db07s696ij_1_1.bkp
File Name: /home/oracle/backup/db08s696iq_1_1.bkp
File Name: /home/oracle/backup/db09s696it_1_1.bkp

List of Files Which Were Not Cataloged
=======================================
File Name: /home/oracle/backup/res.rmn
  RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/backup/pfile.ora
  RMAN-07517: Reason: The file header is corrupted

RMAN> run {
2> restore database;
3> recover database;
4> }
Starting restore at 08-JUN-17
using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /ssdoracle/oradata/STD12CDB/datafile/o1_mf_system_dmj1qxvl_.dbf
channel ORA_DISK_1: restoring datafile 00003 to /ssdoracle/oradata/STD12CDB/datafile/o1_mf_sysaux_dmj1r5mo_.dbf
channel ORA_DISK_1: restoring datafile 00005 to /ssdoracle/oradata/STD12CDB/datafile/o1_mf_undotbs1_dmj1r9qh_.dbf
channel ORA_DISK_1: restoring datafile 00006 to /ssdoracle/oradata/STD12CDB/datafile/o1_mf_users_dmj1rx1x_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/db07s696ij_1_1.bkp
channel ORA_DISK_1: piece handle=/home/oracle/backup/db07s696ij_1_1.bkp tag=TAG20170608T102003
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00002 to /ssdoracle/oradata/STD12CDB/516028AF3ECC7D6DE0534200A8C08E0E/datafile/o1_mf_system_dmj1r1jc_.dbf
channel ORA_DISK_1: restoring datafile 00004 to /ssdoracle/oradata/STD12CDB/516028AF3ECC7D6DE0534200A8C08E0E/datafile/o1_mf_sysaux_dmj1r80p_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/db08s696iq_1_1.bkp
channel ORA_DISK_1: piece handle=/home/oracle/backup/db08s696iq_1_1.bkp tag=TAG20170608T102003
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00027 to /ssdoracle/oradata/STD12CDB/5170445434C83D39E0534200A8C0B781/datafile/o1_mf_system_dml59x8q_.dbf
channel ORA_DISK_1: restoring datafile 00028 to /ssdoracle/oradata/STD12CDB/5170445434C83D39E0534200A8C0B781/datafile/o1_mf_sysaux_dml59x8r_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/db09s696it_1_1.bkp
channel ORA_DISK_1: piece handle=/home/oracle/backup/db09s696it_1_1.bkp tag=TAG20170608T102003
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:15
Finished restore at 08-JUN-17

Starting recover at 08-JUN-17
using channel ORA_DISK_1

starting media recovery

RMAN-08187: warning: media recovery until SCN 7924317 complete
Finished recover at 08-JUN-17

3. Instead of the "open resetlogs" open the database using "open upgrade". The errors are ignorable.

RMAN>  alter database open resetlogs upgrade;

Statement processed
RMAN-06900: warning: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT row
RMAN-06901: warning: disabling update of the V$RMAN_STATUS and V$RMAN_OUTPUT rows
ORACLE error from target database:
ORA-04063: package body "SYS.DBMS_BACKUP_RESTORE" has errors
ORA-06508: PL/SQL: could not find program unit being called: "SYS.DBMS_BACKUP_RESTORE"

4. The root CDB will be open in migrate mode. Open the PDB in migrate mode as well.

SQL> select open_mode from v$database;

OPEN_MODE
-----------
READ WRITE

SQL> select status from v$instance;

STATUS
------------
OPEN MIGRATE

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       MIGRATE    YES
         3 IRPDB                          MOUNTED
SQL> alter pluggable database irpdb open upgrade;

Pluggable database altered.

SQL> show pdbs;

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       MIGRATE    YES
         3 IRPDB                          MIGRATE    YES

5. From this point onwards the steps to upgrade are similar to that of a manual upgrade of a CDB. 12.2 provides a single upgrade script called "dbupgrade" to upgrade the CDB and all or subset of PDBs. In this case all the PDBs (there's only one as this is a SE2) will be upgraded. The upgrade output is given below.

$ which dbupgrade
/opt/app/oracle/product/12.2.0/sehome_1/bin/dbupgrade

$ dbupgrade

Argument list for [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catctl.pl]
Run in                c = 0
Do not run in         C = 0
Input Directory       d = 0
Echo OFF              e = 1
Simulate              E = 0
Forced cleanup        F = 0
Log Id                i = 0
Child Process         I = 0
Log Dir               l = 0
Priority List Name    L = 0
Upgrade Mode active   M = 0
SQL Process Count     n = 0
SQL PDB Process Count N = 0
Open Mode Normal      o = 0
Start Phase           p = 0
End Phase             P = 0
Reverse Order         r = 0
AutoUpgrade Resume    R = 0
Script                s = 0
Serial Run            S = 0
RO User Tablespaces   T = 0
Display Phases        y = 0
Debug catcon.pm       z = 0
Debug catctl.pl       Z = 0

catctl.pl VERSION: [12.2.0.1.0]
           STATUS: [production]
            BUILD: [RDBMS_12.2.0.1.0_LINUX.X64_170125]


/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/orahome = [/opt/app/oracle/product/12.2.0/sehome_1]
/opt/app/oracle/product/12.2.0/sehome_1/bin/orabasehome = [/opt/app/oracle/product/12.2.0/sehome_1]
catctlGetOrabase = [/opt/app/oracle/product/12.2.0/sehome_1]

Analyzing file /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catupgrd.sql

Log file directory = [/tmp/cfgtoollogs/upgrade20170608103851]

catcon: ALL catcon-related output will be written to [/tmp/cfgtoollogs/upgrade20170608103851/catupgrd_catcon_24648.lst]
catcon: See [/tmp/cfgtoollogs/upgrade20170608103851/catupgrd*.log] files for output generated by scripts
catcon: See [/tmp/cfgtoollogs/upgrade20170608103851/catupgrd_*.lst] files for spool files, if any

Number of Cpus        = 8
Database Name         = std12cdb
DataBase Version      = 12.1.0.2.0
catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrd_catcon_24648.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrd*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrd_*.lst] files for spool files, if any

Log file directory = [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852]

Parallel SQL Process Count (PDB)      = 2
Parallel SQL Process Count (CDB$ROOT) = 8
Concurrent PDB Upgrades               = 4
Generated PDB Inclusion:[PDB$SEED IRPDB]
Components in [CDB$ROOT]
    Installed [CATALOG CATPROC CONTEXT XDB]
Not Installed [APEX APS CATJAVA DV EM JAVAVM MGW ODM OLS ORDIM OWM RAC SDO WK XML XOQ]

------------------------------------------------------
Phases [0-115]         Start Time:[2017_06_08 10:38:53]
Container Lists Inclusion:[CDB$ROOT] Exclusion:[NONE]
------------------------------------------------------
***********   Executing Change Scripts   ***********
Serial   Phase #:0    [CDB$ROOT] Files:1    Time: 52s
***************   Catalog Core SQL   ***************
Serial   Phase #:1    [CDB$ROOT] Files:5    Time: 114s
Restart  Phase #:2    [CDB$ROOT] Files:1    Time: 0s
***********   Catalog Tables and Views   ***********
Parallel Phase #:3    [CDB$ROOT] Files:19   Time: 38s
Restart  Phase #:4    [CDB$ROOT] Files:1    Time: 1s
*************   Catalog Final Scripts   ************
Serial   Phase #:5    [CDB$ROOT] Files:6    Time: 41s
*****************   Catproc Start   ****************
Serial   Phase #:6    [CDB$ROOT] Files:1    Time: 24s
*****************   Catproc Types   ****************
Serial   Phase #:7    [CDB$ROOT] Files:2    Time: 20s
Restart  Phase #:8    [CDB$ROOT] Files:1    Time: 1s
****************   Catproc Tables   ****************
Parallel Phase #:9    [CDB$ROOT] Files:69   Time: 44s
Restart  Phase #:10   [CDB$ROOT] Files:1    Time: 0s
*************   Catproc Package Specs   ************
Serial   Phase #:11   [CDB$ROOT] Files:1    Time: 83s
Restart  Phase #:12   [CDB$ROOT] Files:1    Time: 0s
**************   Catproc Procedures   **************
Parallel Phase #:13   [CDB$ROOT] Files:97   Time: 10s
Restart  Phase #:14   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:15   [CDB$ROOT] Files:118  Time: 18s
Restart  Phase #:16   [CDB$ROOT] Files:1    Time: 1s
Serial   Phase #:17   [CDB$ROOT] Files:13   Time: 4s
Restart  Phase #:18   [CDB$ROOT] Files:1    Time: 0s
*****************   Catproc Views   ****************
Parallel Phase #:19   [CDB$ROOT] Files:33   Time: 38s
Restart  Phase #:20   [CDB$ROOT] Files:1    Time: 1s
Serial   Phase #:21   [CDB$ROOT] Files:3    Time: 11s
Restart  Phase #:22   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:23   [CDB$ROOT] Files:24   Time: 90s
Restart  Phase #:24   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:25   [CDB$ROOT] Files:11   Time: 56s
Restart  Phase #:26   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:27   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:28   [CDB$ROOT] Files:3    Time: 5s
Serial   Phase #:29   [CDB$ROOT] Files:1    Time: 0s
Restart  Phase #:30   [CDB$ROOT] Files:1    Time: 0s
***************   Catproc CDB Views   **************
Serial   Phase #:31   [CDB$ROOT] Files:1    Time: 1s
Restart  Phase #:32   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:34   [CDB$ROOT] Files:1    Time: 0s
*****************   Catproc PLBs   *****************
Serial   Phase #:35   [CDB$ROOT] Files:283  Time: 27s
Serial   Phase #:36   [CDB$ROOT] Files:1    Time: 0s
Restart  Phase #:37   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:38   [CDB$ROOT] Files:1    Time: 4s
Restart  Phase #:39   [CDB$ROOT] Files:1    Time: 0s
***************   Catproc DataPump   ***************
Serial   Phase #:40   [CDB$ROOT] Files:3    Time: 60s
Restart  Phase #:41   [CDB$ROOT] Files:1    Time: 0s
******************   Catproc SQL   *****************
Parallel Phase #:42   [CDB$ROOT] Files:13   Time: 61s
Restart  Phase #:43   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:44   [CDB$ROOT] Files:12   Time: 10s
Restart  Phase #:45   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:46   [CDB$ROOT] Files:2    Time: 1s
Restart  Phase #:47   [CDB$ROOT] Files:1    Time: 0s
*************   Final Catproc scripts   ************
Serial   Phase #:48   [CDB$ROOT] Files:1    Time: 7s
Restart  Phase #:49   [CDB$ROOT] Files:1    Time: 0s
**************   Final RDBMS scripts   *************
Serial   Phase #:50   [CDB$ROOT] Files:1    Time: 35s
************   Upgrade Component Start   ***********
Serial   Phase #:51   [CDB$ROOT] Files:1    Time: 0s
Restart  Phase #:52   [CDB$ROOT] Files:1    Time: 1s
****************   Upgrading Java   ****************
Serial   Phase #:53   [CDB$ROOT] Files:1    Time: 0s
Restart  Phase #:54   [CDB$ROOT] Files:1    Time: 0s
*****************   Upgrading XDK   ****************
Serial   Phase #:55   [CDB$ROOT] Files:1    Time: 1s
Restart  Phase #:56   [CDB$ROOT] Files:1    Time: 0s
*********   Upgrading APS,OLS,DV,CONTEXT   *********
Serial   Phase #:57   [CDB$ROOT] Files:1    Time: 51s
*****************   Upgrading XDB   ****************
Restart  Phase #:58   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:60   [CDB$ROOT] Files:3    Time: 11s
Serial   Phase #:61   [CDB$ROOT] Files:3    Time: 8s
Parallel Phase #:62   [CDB$ROOT] Files:9    Time: 2s
Parallel Phase #:63   [CDB$ROOT] Files:24   Time: 3s
Serial   Phase #:64   [CDB$ROOT] Files:4    Time: 11s
Serial   Phase #:65   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:66   [CDB$ROOT] Files:30   Time: 4s
Serial   Phase #:67   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:68   [CDB$ROOT] Files:6    Time: 2s
Serial   Phase #:69   [CDB$ROOT] Files:2    Time: 19s
Serial   Phase #:70   [CDB$ROOT] Files:3    Time: 36s
Restart  Phase #:71   [CDB$ROOT] Files:1    Time: 0s
*********   Upgrading CATJAVA,OWM,MGW,RAC   ********
Serial   Phase #:72   [CDB$ROOT] Files:1    Time: 1s
****************   Upgrading ORDIM   ***************
Restart  Phase #:73   [CDB$ROOT] Files:1    Time: 1s
Serial   Phase #:75   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:76   [CDB$ROOT] Files:2    Time: 0s
Serial   Phase #:77   [CDB$ROOT] Files:1    Time: 1s
Restart  Phase #:78   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:79   [CDB$ROOT] Files:2    Time: 0s
Serial   Phase #:80   [CDB$ROOT] Files:2    Time: 1s
*****************   Upgrading SDO   ****************
Restart  Phase #:81   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:83   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:84   [CDB$ROOT] Files:1    Time: 1s
Restart  Phase #:85   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:86   [CDB$ROOT] Files:1    Time: 0s
Restart  Phase #:87   [CDB$ROOT] Files:1    Time: 1s
Parallel Phase #:88   [CDB$ROOT] Files:3    Time: 0s
Restart  Phase #:89   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:90   [CDB$ROOT] Files:1    Time: 1s
Restart  Phase #:91   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:92   [CDB$ROOT] Files:1    Time: 0s
Restart  Phase #:93   [CDB$ROOT] Files:1    Time: 0s
Parallel Phase #:94   [CDB$ROOT] Files:4    Time: 1s
Restart  Phase #:95   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:96   [CDB$ROOT] Files:1    Time: 1s
Restart  Phase #:97   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:98   [CDB$ROOT] Files:2    Time: 0s
Restart  Phase #:99   [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:100  [CDB$ROOT] Files:1    Time: 1s
Restart  Phase #:101  [CDB$ROOT] Files:1    Time: 0s
***********   Upgrading Misc. ODM, OLAP   **********
Serial   Phase #:102  [CDB$ROOT] Files:1    Time: 0s
****************   Upgrading APEX   ****************
Restart  Phase #:103  [CDB$ROOT] Files:1    Time: 1s
Serial   Phase #:104  [CDB$ROOT] Files:1    Time: 0s
Restart  Phase #:105  [CDB$ROOT] Files:1    Time: 0s
***********   Final Component scripts    ***********
Serial   Phase #:106  [CDB$ROOT] Files:1    Time: 1s
*************   Final Upgrade scripts   ************
Serial   Phase #:107  [CDB$ROOT] Files:1    Time: 85s
**********   End PDB Application Upgrade   *********
Serial   Phase #:108  [CDB$ROOT] Files:1    Time: 0s
*******************   Migration   ******************
Serial   Phase #:109  [CDB$ROOT] Files:1    Time: 36s
Serial   Phase #:110  [CDB$ROOT] Files:1    Time: 1s
Serial   Phase #:111  [CDB$ROOT] Files:1    Time: 45s
*****************   Post Upgrade   *****************
Serial   Phase #:112  [CDB$ROOT] Files:1    Time: 13s
****************   Summary report   ****************
Serial   Phase #:113  [CDB$ROOT] Files:1    Time: 1s
Serial   Phase #:114  [CDB$ROOT] Files:1    Time: 0s
Serial   Phase #:115  [CDB$ROOT] Files:1     Time: 0s

------------------------------------------------------
Phases [0-115]         End Time:[2017_06_08 10:58:53]
Container Lists Inclusion:[CDB$ROOT] Exclusion:[NONE]
------------------------------------------------------

Start processing of PDB$SEED
[/opt/app/oracle/product/12.2.0/sehome_1/perl/bin/perl /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catctl.pl -I -i pdb_seed -n 2 -c 'PDB$SEED' -l /opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852 /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catupgrd.sql]

Start processing of IRPDB
[/opt/app/oracle/product/12.2.0/sehome_1/perl/bin/perl /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catctl.pl -I -i irpdb -n 2 -c 'IRPDB' -l /opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852 /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catupgrd.sql]

Argument list for [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catctl.pl]
Run in                c = PDB$SEED
Do not run in         C = 0
Input Directory       d = 0
Echo OFF              e = 1
Simulate              E = 0
Forced cleanup        F = 0
Log Id                i = pdb_seed
Child Process         I = 1
Log Dir               l = /opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852
Priority List Name    L = 0
Upgrade Mode active   M = 0
SQL Process Count     n = 2
SQL PDB Process Count N = 0
Open Mode Normal      o = 0
Start Phase           p = 0
End Phase             P = 0
Reverse Order         r = 0
AutoUpgrade Resume    R = 0
Script                s = 0
Serial Run            S = 0
RO User Tablespaces   T = 0
Display Phases        y = 0
Debug catcon.pm       z = 0
Debug catctl.pl       Z = 0

catctl.pl VERSION: [12.2.0.1.0]
           STATUS: [production]
            BUILD: [RDBMS_12.2.0.1.0_LINUX.X64_170125]



Argument list for [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catctl.pl]
Run in                c = IRPDB
Do not run in         C = 0
Input Directory       d = 0
Echo OFF              e = 1
Simulate              E = 0
Forced cleanup        F = 0
Log Id                i = irpdb
Child Process         I = 1
Log Dir               l = /opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852
Priority List Name    L = 0
Upgrade Mode active   M = 0
SQL Process Count     n = 2
SQL PDB Process Count N = 0
Open Mode Normal      o = 0
Start Phase           p = 0
End Phase             P = 0
Reverse Order         r = 0
AutoUpgrade Resume    R = 0
Script                s = 0
Serial Run            S = 0
RO User Tablespaces   T = 0
Display Phases        y = 0
Debug catcon.pm       z = 0
Debug catctl.pl       Z = 0

catctl.pl VERSION: [12.2.0.1.0]
           STATUS: [production]
            BUILD: [RDBMS_12.2.0.1.0_LINUX.X64_170125]


/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/orahome = [/opt/app/oracle/product/12.2.0/sehome_1]
/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/orahome = [/opt/app/oracle/product/12.2.0/sehome_1]
/opt/app/oracle/product/12.2.0/sehome_1/bin/orabasehome = [/opt/app/oracle/product/12.2.0/sehome_1]
catctlGetOrabase = [/opt/app/oracle/product/12.2.0/sehome_1]

Analyzing file /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catupgrd.sql

Log file directory = [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852]

/opt/app/oracle/product/12.2.0/sehome_1/bin/orabasehome = [/opt/app/oracle/product/12.2.0/sehome_1]
catctlGetOrabase = [/opt/app/oracle/product/12.2.0/sehome_1]

Analyzing file /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catupgrd.sql

Log file directory = [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852]

catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdpdb_seed_catcon_581.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdpdb_seed*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdpdb_seed_*.lst] files for spool files, if any
catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdirpdb_catcon_585.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdirpdb*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdirpdb_*.lst] files for spool files, if any

Number of Cpus        = 8

Number of Cpus        = 8
Database Name         = std12cdb
Database Name         = std12cdb
DataBase Version      = 12.2.0.1.0
DataBase Version      = 12.2.0.1.0
Generated PDB Inclusion:[IRPDB]
CDB$ROOT  Open Mode = [OPEN]
Generated PDB Inclusion:[PDB$SEED]
Components in [IRPDB]
    Installed [CATALOG CATPROC CONTEXT XDB]
Not Installed [APEX APS CATJAVA DV EM JAVAVM MGW ODM OLS ORDIM OWM RAC SDO WK XML XOQ]

------------------------------------------------------
Phases [0-115]         Start Time:[2017_06_08 10:58:58]
Container Lists Inclusion:[IRPDB] Exclusion:[NONE]
------------------------------------------------------
CDB$ROOT  Open Mode = [OPEN]
Components in [PDB$SEED]
    Installed [CATALOG CATPROC CONTEXT XDB]
Not Installed [APEX APS CATJAVA DV EM JAVAVM MGW ODM OLS ORDIM OWM RAC SDO WK XML XOQ]

------------------------------------------------------
Phases [0-115]         Start Time:[2017_06_08 10:58:59]
Container Lists Inclusion:[PDB$SEED] Exclusion:[NONE]
------------------------------------------------------
***********   Executing Change Scripts   ***********
Serial   Phase #:0    [IRPDB] Files:1 ***********   Executing Change Scripts   ***********
Serial   Phase #:0    [PDB$SEED] Files:1    Time: 74s
***************   Catalog Core SQL   ***************
Serial   Phase #:1    [PDB$SEED] Files:5    Time: 74s
***************   Catalog Core SQL   ***************
Serial   Phase #:1    [IRPDB] Files:5    Time: 215s
Restart  Phase #:2    [PDB$SEED] Files:1    Time: 215s
Restart  Phase #:2    [IRPDB] Files:1    Time: 0s
***********   Catalog Tables and Views   ***********
Parallel Phase #:3    [IRPDB] Files:19    Time: 0s
***********   Catalog Tables and Views   ***********
Parallel Phase #:3    [PDB$SEED] Files:19   Time: 104s
Restart  Phase #:4    [PDB$SEED] Files:1   Time: 104s
Restart  Phase #:4    [IRPDB] Files:1    Time: 1s
*************   Catalog Final Scripts   ************
Serial   Phase #:5    [PDB$SEED] Files:6    Time: 1s
*************   Catalog Final Scripts   ************
Serial   Phase #:5    [IRPDB] Files:6    Time: 76s
*****************   Catproc Start   ****************
Serial   Phase #:6    [IRPDB] Files:1    Time: 76s
*****************   Catproc Start   ****************
Serial   Phase #:6    [PDB$SEED] Files:1    Time: 38s
*****************   Catproc Types   ****************
Serial   Phase #:7    [PDB$SEED] Files:2    Time: 38s
*****************   Catproc Types   ****************
Serial   Phase #:7    [IRPDB] Files:2    Time: 29s
Restart  Phase #:8    [IRPDB] Files:1    Time: 29s
Restart  Phase #:8    [PDB$SEED] Files:1    Time: 1s
****************   Catproc Tables   ****************
Parallel Phase #:9    [IRPDB] Files:69    Time: 1s
****************   Catproc Tables   ****************
Parallel Phase #:9    [PDB$SEED] Files:69   Time: 113s
Restart  Phase #:10   [IRPDB] Files:1   Time: 113s
Restart  Phase #:10   [PDB$SEED] Files:1    Time: 0s
*************   Catproc Package Specs   ************
Serial   Phase #:11   [PDB$SEED] Files:1    Time: 0s
*************   Catproc Package Specs   ************
Serial   Phase #:11   [IRPDB] Files:1    Time: 133s
Restart  Phase #:12   [PDB$SEED] Files:1    Time: 133s
Restart  Phase #:12   [IRPDB] Files:1    Time: 0s
**************   Catproc Procedures   **************
Parallel Phase #:13   [IRPDB] Files:97    Time: 0s
**************   Catproc Procedures   **************
Parallel Phase #:13   [PDB$SEED] Files:97   Time: 16s
Restart  Phase #:14   [IRPDB] Files:1   Time: 16s
Restart  Phase #:14   [PDB$SEED] Files:1    Time: 0s
Parallel Phase #:15   [IRPDB] Files:118    Time: 0s
Parallel Phase #:15   [PDB$SEED] Files:118  Time: 26s
Restart  Phase #:16   [IRPDB] Files:1  Time: 27s
Restart  Phase #:16   [PDB$SEED] Files:1    Time: 1s
Serial   Phase #:17   [IRPDB] Files:13    Time: 0s
Serial   Phase #:17   [PDB$SEED] Files:13   Time: 6s
Restart  Phase #:18   [IRPDB] Files:1   Time: 6s
Restart  Phase #:18   [PDB$SEED] Files:1    Time: 0s
   Time: 0s
*****************   Catproc Views   ****************
*****************   Catproc Views   ****************
Parallel Phase #:19   [PDB$SEED] Files:33 Parallel Phase #:19   [IRPDB] Files:33   Time: 75s
Restart  Phase #:20   [PDB$SEED] Files:1   Time: 75s
Restart  Phase #:20   [IRPDB] Files:1    Time: 0s
Serial   Phase #:21   [IRPDB] Files:3    Time: 0s
Serial   Phase #:21   [PDB$SEED] Files:3    Time: 24s
Restart  Phase #:22   [PDB$SEED] Files:1    Time: 24s
Restart  Phase #:22   [IRPDB] Files:1    Time: 0s
Parallel Phase #:23   [PDB$SEED] Files:24    Time: 0s
Parallel Phase #:23   [IRPDB] Files:24   Time: 232s
Restart  Phase #:24   [PDB$SEED] Files:1   Time: 232s
Restart  Phase #:24   [IRPDB] Files:1    Time: 1s
Parallel Phase #:25   [IRPDB] Files:11    Time: 1s
Parallel Phase #:25   [PDB$SEED] Files:11   Time: 77s
Restart  Phase #:26   [PDB$SEED] Files:1   Time: 77s
Restart  Phase #:26   [IRPDB] Files:1    Time: 0s
Serial   Phase #:27   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:28   [PDB$SEED] Files:3    Time: 0s
Serial   Phase #:27   [IRPDB] Files:1    Time: 0s
Serial   Phase #:28   [IRPDB] Files:3    Time: 29s
Serial   Phase #:29   [PDB$SEED] Files:1    Time: 0s
Restart  Phase #:30   [PDB$SEED] Files:1    Time: 29s
Serial   Phase #:29   [IRPDB] Files:1    Time: 0s
Restart  Phase #:30   [IRPDB] Files:1    Time: 0s
***************   Catproc CDB Views   **************
Serial   Phase #:31   [PDB$SEED] Files:1    Time: 0s
***************   Catproc CDB Views   **************
Serial   Phase #:31   [IRPDB] Files:1    Time: 1s
Restart  Phase #:32   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:32   [IRPDB] Files:1    Time: 0s
Serial   Phase #:34   [PDB$SEED] Files:1    Time: 0s
*****************   Catproc PLBs   *****************
Serial   Phase #:35   [PDB$SEED] Files:283    Time: 0s
Serial   Phase #:34   [IRPDB] Files:1    Time: 0s
*****************   Catproc PLBs   *****************
Serial   Phase #:35   [IRPDB] Files:283  Time: 196s
Serial   Phase #:36   [IRPDB] Files:1    Time: 0s
Restart  Phase #:37   [IRPDB] Files:1  Time: 196s
Serial   Phase #:36   [PDB$SEED] Files:1    Time: 0s
Restart  Phase #:37   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:38   [IRPDB] Files:1    Time: 0s
Serial   Phase #:38   [PDB$SEED] Files:1    Time: 7s
Restart  Phase #:39   [PDB$SEED] Files:1    Time: 7s
Restart  Phase #:39   [IRPDB] Files:1    Time: 1s
***************   Catproc DataPump   ***************
Serial   Phase #:40   [PDB$SEED] Files:3    Time: 1s
***************   Catproc DataPump   ***************
Serial   Phase #:40   [IRPDB] Files:3    Time: 61s
Restart  Phase #:41   [IRPDB] Files:1    Time: 0s
******************   Catproc SQL   *****************
Parallel Phase #:42   [IRPDB] Files:13    Time: 61s
Restart  Phase #:41   [PDB$SEED] Files:1    Time: 0s
******************   Catproc SQL   *****************
Parallel Phase #:42   [PDB$SEED] Files:13   Time: 75s
Restart  Phase #:43   [PDB$SEED] Files:1    Time: 0s
Parallel Phase #:44   [PDB$SEED] Files:12   Time: 76s
Restart  Phase #:43   [IRPDB] Files:1    Time: 0s
Parallel Phase #:44   [IRPDB] Files:12   Time: 8s
Restart  Phase #:45   [IRPDB] Files:1   Time: 9s
Restart  Phase #:45   [PDB$SEED] Files:1    Time: 0s
Parallel Phase #:46   [IRPDB] Files:2    Time: 0s
Parallel Phase #:46   [PDB$SEED] Files:2    Time: 1s
Restart  Phase #:47   [IRPDB] Files:1    Time: 1s
Restart  Phase #:47   [PDB$SEED] Files:1    Time: 0s
*************   Final Catproc scripts   ************
Serial   Phase #:48   [IRPDB] Files:1    Time: 0s
*************   Final Catproc scripts   ************
Serial   Phase #:48   [PDB$SEED] Files:1    Time: 15s
Restart  Phase #:49   [IRPDB] Files:1    Time: 15s
Restart  Phase #:49   [PDB$SEED] Files:1    Time: 0s
**************   Final RDBMS scripts   *************
Serial   Phase #:50   [IRPDB] Files:1    Time: 0s
**************   Final RDBMS scripts   *************
Serial   Phase #:50   [PDB$SEED] Files:1    Time: 35s
************   Upgrade Component Start   ***********
Serial   Phase #:51   [PDB$SEED] Files:1    Time: 35s
************   Upgrade Component Start   ***********
Serial   Phase #:51   [IRPDB] Files:1    Time: 1s
Restart  Phase #:52   [IRPDB] Files:1    Time: 1s
Restart  Phase #:52   [PDB$SEED] Files:1    Time: 0s
****************   Upgrading Java   ****************
Serial   Phase #:53   [PDB$SEED] Files:1    Time: 0s
****************   Upgrading Java   ****************
Serial   Phase #:53   [IRPDB] Files:1    Time: 1s
Restart  Phase #:54   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:54   [IRPDB] Files:1    Time: 0s
*****************   Upgrading XDK   ****************
Serial   Phase #:55   [PDB$SEED] Files:1    Time: 0s
*****************   Upgrading XDK   ****************
Serial   Phase #:55   [IRPDB] Files:1    Time: 0s
Restart  Phase #:56   [IRPDB] Files:1    Time: 1s
Restart  Phase #:56   [PDB$SEED] Files:1    Time: 1s
*********   Upgrading APS,OLS,DV,CONTEXT   *********
Serial   Phase #:57   [IRPDB] Files:1    Time: 0s
*********   Upgrading APS,OLS,DV,CONTEXT   *********
Serial   Phase #:57   [PDB$SEED] Files:1    Time: 58s
*****************   Upgrading XDB   ****************
Restart  Phase #:58   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:60   [PDB$SEED] Files:3    Time: 59s
*****************   Upgrading XDB   ****************
Restart  Phase #:58   [IRPDB] Files:1    Time: 0s
Serial   Phase #:60   [IRPDB] Files:3    Time: 14s
Serial   Phase #:61   [PDB$SEED] Files:3    Time: 13s
Serial   Phase #:61   [IRPDB] Files:3    Time: 5s
   Time: 5s
Parallel Phase #:62   [IRPDB] Files:9 Parallel Phase #:62   [PDB$SEED] Files:9    Time: 2s
Parallel Phase #:63   [IRPDB] Files:24    Time: 2s
Parallel Phase #:63   [PDB$SEED] Files:24   Time: 7s
Serial   Phase #:64   [IRPDB] Files:4   Time: 7s
Serial   Phase #:64   [PDB$SEED] Files:4    Time: 26s
Serial   Phase #:65   [IRPDB] Files:1    Time: 0s
Serial   Phase #:66   [IRPDB] Files:30    Time: 26s
Serial   Phase #:65   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:66   [PDB$SEED] Files:30   Time: 13s
Serial   Phase #:67   [IRPDB] Files:1    Time: 0s
Parallel Phase #:68   [IRPDB] Files:6   Time: 13s
Serial   Phase #:67   [PDB$SEED] Files:1    Time: 0s
Parallel Phase #:68   [PDB$SEED] Files:6    Time: 4s
Serial   Phase #:69   [PDB$SEED] Files:2    Time: 4s
Serial   Phase #:69   [IRPDB] Files:2    Time: 21s
Serial   Phase #:70   [IRPDB] Files:3    Time: 21s
Serial   Phase #:70   [PDB$SEED] Files:3    Time: 44s
Restart  Phase #:71   [PDB$SEED] Files:1    Time: 44s
Restart  Phase #:71   [IRPDB] Files:1    Time: 0s
*********   Upgrading CATJAVA,OWM,MGW,RAC   ********
Serial   Phase #:72   [IRPDB] Files:1    Time: 0s
*********   Upgrading CATJAVA,OWM,MGW,RAC   ********
Serial   Phase #:72   [PDB$SEED] Files:1    Time: 1s
****************   Upgrading ORDIM   ***************
Restart  Phase #:73   [PDB$SEED] Files:1    Time: 1s
****************   Upgrading ORDIM   ***************
Restart  Phase #:73   [IRPDB] Files:1    Time: 0s
Serial   Phase #:75   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:75   [IRPDB] Files:1    Time: 1s
Parallel Phase #:76   [PDB$SEED] Files:2    Time: 1s
Parallel Phase #:76   [IRPDB] Files:2    Time: 1s
Serial   Phase #:77   [PDB$SEED] Files:1    Time: 1s
Serial   Phase #:77   [IRPDB] Files:1    Time: 1s
Restart  Phase #:78   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:78   [IRPDB] Files:1    Time: 0s
Parallel Phase #:79   [PDB$SEED] Files:2    Time: 0s
Parallel Phase #:79   [IRPDB] Files:2    Time: 0s
Serial   Phase #:80   [PDB$SEED] Files:2    Time: 1s
Serial   Phase #:80   [IRPDB] Files:2    Time: 1s
*****************   Upgrading SDO   ****************
Restart  Phase #:81   [PDB$SEED] Files:1    Time: 0s
*****************   Upgrading SDO   ****************
Restart  Phase #:81   [IRPDB] Files:1    Time: 0s
Serial   Phase #:83   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:83   [IRPDB] Files:1    Time: 1s
Serial   Phase #:84   [PDB$SEED] Files:1    Time: 1s
Serial   Phase #:84   [IRPDB] Files:1    Time: 1s
Restart  Phase #:85   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:85   [IRPDB] Files:1    Time: 0s
Serial   Phase #:86   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:86   [IRPDB] Files:1    Time: 1s
Restart  Phase #:87   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:87   [IRPDB] Files:1    Time: 0s
Parallel Phase #:88   [PDB$SEED] Files:3    Time: 0s
Parallel Phase #:88   [IRPDB] Files:3    Time: 0s
Restart  Phase #:89   [PDB$SEED] Files:1    Time: 0s
Restart  Phase #:89   [IRPDB] Files:1    Time: 1s
Serial   Phase #:90   [PDB$SEED] Files:1    Time: 1s
Serial   Phase #:90   [IRPDB] Files:1    Time: 0s
Restart  Phase #:91   [PDB$SEED] Files:1    Time: 0s
Restart  Phase #:91   [IRPDB] Files:1    Time: 0s
Serial   Phase #:92   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:92   [IRPDB] Files:1    Time: 1s
Restart  Phase #:93   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:93   [IRPDB] Files:1    Time: 0s
Parallel Phase #:94   [PDB$SEED] Files:4    Time: 0s
Parallel Phase #:94   [IRPDB] Files:4    Time: 1s
Restart  Phase #:95   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:95   [IRPDB] Files:1    Time: 0s
Serial   Phase #:96   [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:96   [IRPDB] Files:1    Time: 1s
Restart  Phase #:97   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:97   [IRPDB] Files:1    Time: 0s
Serial   Phase #:98   [PDB$SEED] Files:2    Time: 0s
Serial   Phase #:98   [IRPDB] Files:2    Time: 1s
Restart  Phase #:99   [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:99   [IRPDB] Files:1    Time: 0s
Serial   Phase #:100  [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:100  [IRPDB] Files:1    Time: 0s
Restart  Phase #:101  [PDB$SEED] Files:1    Time: 0s
Restart  Phase #:101  [IRPDB] Files:1    Time: 0s
***********   Upgrading Misc. ODM, OLAP   **********
Serial   Phase #:102  [PDB$SEED] Files:1    Time: 0s
***********   Upgrading Misc. ODM, OLAP   **********
Serial   Phase #:102  [IRPDB] Files:1    Time: 1s
****************   Upgrading APEX   ****************
Restart  Phase #:103  [PDB$SEED] Files:1    Time: 1s
****************   Upgrading APEX   ****************
Restart  Phase #:103  [IRPDB] Files:1    Time: 0s
Serial   Phase #:104  [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:104  [IRPDB] Files:1    Time: 1s
Restart  Phase #:105  [PDB$SEED] Files:1    Time: 1s
Restart  Phase #:105  [IRPDB] Files:1    Time: 0s
***********   Final Component scripts    ***********
Serial   Phase #:106  [PDB$SEED] Files:1    Time: 0s
***********   Final Component scripts    ***********
Serial   Phase #:106  [IRPDB] Files:1    Time: 1s
*************   Final Upgrade scripts   ************
Serial   Phase #:107  [PDB$SEED] Files:1    Time: 1s
*************   Final Upgrade scripts   ************
Serial   Phase #:107  [IRPDB] Files:1    Time: 107s
**********   End PDB Application Upgrade   *********
Serial   Phase #:108  [PDB$SEED] Files:1    Time: 1s
*******************   Migration   ******************
Serial   Phase #:109  [PDB$SEED] Files:1    Time: 111s
**********   End PDB Application Upgrade   *********
Serial   Phase #:108  [IRPDB] Files:1    Time: 1s
*******************   Migration   ******************
Serial   Phase #:109  [IRPDB] Files:1    Time: 46s
Serial   Phase #:110  [PDB$SEED] Files:1    Time: 3s
Serial   Phase #:111  [PDB$SEED] Files:1    Time: 45s
Serial   Phase #:110  [IRPDB] Files:1    Time: 2s
Serial   Phase #:111  [IRPDB] Files:1    Time: 10s
*****************   Post Upgrade   *****************
Serial   Phase #:112  [PDB$SEED] Files:1    Time: 15s
*****************   Post Upgrade   *****************
Serial   Phase #:112  [IRPDB] Files:1    Time: 90s
****************   Summary report   ****************
Serial   Phase #:113  [IRPDB] Files:1    Time: 1s
Serial   Phase #:114  [IRPDB] Files:1    Time: 2s
Serial   Phase #:115  [IRPDB] Files:1     Time: 0s

------------------------------------------------------
Phases [0-115]         End Time:[2017_06_08 11:34:51]
Container Lists Inclusion:[IRPDB] Exclusion:[NONE]
------------------------------------------------------

Grand Total Time: 2154s [IRPDB]

 LOG FILES: (/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdirpdb*.log)

Upgrade Summary Report Located in:
/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/upg_summary.log

Total Upgrade Time:          [0d:0h:35m:54s]

   Time: 475s
****************   Summary report   ****************
Serial   Phase #:113  [PDB$SEED] Files:1    Time: 0s
Serial   Phase #:114  [PDB$SEED] Files:1    Time: 3s
Serial   Phase #:115  [PDB$SEED] Files:1     Time: 0s

------------------------------------------------------
Phases [0-115]         End Time:[2017_06_08 11:41:09]
Container Lists Inclusion:[PDB$SEED] Exclusion:[NONE]
------------------------------------------------------

Grand Total Time: 2533s [PDB$SEED]

 LOG FILES: (/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrdpdb_seed*.log)

Upgrade Summary Report Located in:
/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/upg_summary.log

Total Upgrade Time:          [0d:0h:42m:13s]

     Time: 1200s For CDB$ROOT
     Time: 2541s For PDB(s)

Grand Total Time: 3741s

 LOG FILES: (/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/catupgrd*.log)

Upgrade Summary Report Located in:
/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/upg_summary.log

Grand Total Upgrade Time:    [0d:1h:2m:21s]

6. Upgrading summary shows that time-zone requires upgrading for CDB and the PDBs.

$ cat /opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/std12cdb/upgrade20170608103852/upg_summary.log

Oracle Database 12.2 Post-Upgrade Status Tool           06-08-2017 10:58:52
                             [CDB$ROOT]

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                          UPGRADED      12.2.0.1.0  00:14:14
Oracle Real Application Clusters       UPGRADED      12.2.0.1.0  00:00:00
Oracle Text                            UPGRADED      12.2.0.1.0  00:00:50
Oracle XML Database                    UPGRADED      12.2.0.1.0  00:01:35
Final Actions                                                    00:02:01
Post Upgrade                                                     00:00:12

Total Upgrade Time: 00:19:08 [CDB$ROOT]

Database time zone version is 18. It is older than current release time
zone version 26. Time zone upgrade is needed using the DBMS_DST package.


Oracle Database 12.2 Post-Upgrade Status Tool           06-08-2017 11:34:49
                             [IRPDB]

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                          UPGRADED      12.2.0.1.0  00:27:49
Oracle Real Application Clusters       UPGRADED      12.2.0.1.0  00:00:00
Oracle Text                            UPGRADED      12.2.0.1.0  00:00:58
Oracle XML Database                    UPGRADED      12.2.0.1.0  00:02:13
Final Actions                                                    00:02:37
Post Upgrade                                                     00:01:30

Total Upgrade Time: 00:35:28 [IRPDB]

Database time zone version is 18. It is older than current release time
zone version 26. Time zone upgrade is needed using the DBMS_DST package.


Oracle Database 12.2 Post-Upgrade Status Tool           06-08-2017 11:41:06
                             [PDB$SEED]

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                             VALID      12.2.0.1.0  00:27:49
Oracle Real Application Clusters     OPTION OFF      12.2.0.1.0  00:00:00
Oracle Text                               VALID      12.2.0.1.0  00:00:57
Oracle XML Database                       VALID      12.2.0.1.0  00:02:13
Final Actions                                                    00:02:34
Post Upgrade                                                     00:01:29
Post Compile                                                     00:06:22

Total Upgrade Time: 00:41:47 [PDB$SEED]

Database time zone version is 18. It is older than current release time
zone version 26. Time zone upgrade is needed using the DBMS_DST package.


Upgrade Times Sorted In Descending Order

Total Upgrade Time: 00:41:47 [PDB$SEED]
Total Upgrade Time: 00:35:28 [IRPDB]
Total Upgrade Time: 00:19:08 [CDB$ROOT]
Grand Total Upgrade Time:    [0d:1h:2m:21s]

7. Before the timezone upgrade run the utlrp to compile packages.

$ORACLE_HOME/perl/bin/perl catcon.pl -n 1 -e -b utlrp -d '''.''' utlrp.sql
catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/utlrp_catcon_18972.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/utlrp*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/utlrp_*.lst] files for spool files, if any
catcon.pl: completed successfully

8. If timezone is being updated then it must be updated in both CDB$ROOT and the PDBs. Check the timezone upgrade state on CDB and PDBs. Initially the upgrade state will be 0 and secondary timezone will be 0. Refer here for further information on timezone upgrade.

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME; 

PROPERTY_NAME                  VALUE
------------------------------ ----------
DST_PRIMARY_TT_VERSION         18
DST_SECONDARY_TT_VERSION       0
DST_UPGRADE_STATE              NONE

9. Start the CDB and all the PBDs in upgrade mode

SQL> Startup  upgrade;
SQL> alter pluggable database irpdb open upgrade;

Pluggable database altered.

SQL> show pdbs;

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       MIGRATE    YES
         3 IRPDB                          MIGRATE    YES

10. Create a timezone upgrade script to prepare for timezone upgrade and run it using catcon.pl on all the containers.

cat begin_tz.sql
exec DBMS_DST.BEGIN_UPGRADE(26);

$ $ORACLE_HOME/perl/bin/perl catcon.pl -n 1 -e -b begintz -d '''.'''begin_tz.sql
catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/begintz_catcon_14761.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/begintz*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/begintz_*.lst] files for spool files, if any
catcon.pl: completed successfully

11. Shutdown the and start the database in normal mode.

SQL> shutdown immedaite;
SQL> startup;

The upgrade state and primary TT version should have been changed as below. Output is from the PDB execution but it should be same on the CDB as well.

SQL> show con_name

CON_NAME
------------------------------
IRPDB
SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME;  

PROPERTY_NAME                  VALUE
------------------------------ ----------
DST_PRIMARY_TT_VERSION         26
DST_SECONDARY_TT_VERSION       18
DST_UPGRADE_STATE              UPGRADE

12. Create a timezone upgrade script that will be the actual timezone upgrade. Run it using catcon.pl.

$ cat upgrdtz.sql
truncate table sys.dst$error_table;
truncate table sys.dst$trigger_table;

VAR numfail number
BEGIN
DBMS_DST.UPGRADE_DATABASE(:numfail,
parallel                  => TRUE,
log_errors                => TRUE,
log_errors_table          =>'SYS.DST$ERROR_TABLE',
log_triggers_table        =>'SYS.DST$TRIGGER_TABLE',
error_on_overlap_time     => TRUE,
error_on_nonexisting_time => TRUE);
DBMS_OUTPUT.PUT_LINE('Failures:'|| :numfail);
END;
/

exec DBMS_DST.END_UPGRADE(:numfail);

$ORACLE_HOME/perl/bin/perl catcon.pl -n 1 -e -b upgrdtz -d '''.'''upgrdtz.sql
catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/upgrdtz_catcon_15286.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/upgrdtz*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/upgrdtz_*.lst] files for spool files, if any

13. Check if the timezone has been upgraded.

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME;

PROPERTY_NAME                  VALUE
------------------------------ ----------
DST_PRIMARY_TT_VERSION         26
DST_SECONDARY_TT_VERSION       0
DST_UPGRADE_STATE              NONE

SQL> select * from v$timezone_file;

FILENAME                VERSION     CON_ID
-------------------- ---------- ----------
timezlrg_26.dat              26          0

14. Run the post ugprade status tool and verify the upgrade status. The output now shows that timezone meets the current release needs.

$ORACLE_HOME/perl/bin/perl catcon.pl -n 1 -e -b utlu122s -d '''.''' utlu122s.sql

Oracle Database 12.2 Post-Upgrade Status Tool           06-08-2017 12:27:50
[CDB$ROOT]

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                          UPGRADED      12.2.0.1.0  00:14:14
Oracle Real Application Clusters       UPGRADED      12.2.0.1.0  00:00:00
Oracle Text                            UPGRADED      12.2.0.1.0  00:00:50
Oracle XML Database                    UPGRADED      12.2.0.1.0  00:01:35
Final Actions                                                    00:02:01
Post Upgrade                                                     00:00:12

Total Upgrade Time: 00:19:08 [CDB$ROOT]

Database time zone version is 26. It meets current release needs.

Oracle Database 12.2 Post-Upgrade Status Tool           06-08-2017 12:27:52
[PDB$SEED]

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                             VALID      12.2.0.1.0  00:27:49
Oracle Real Application Clusters     OPTION OFF      12.2.0.1.0  00:00:00
Oracle Text                               VALID      12.2.0.1.0  00:00:57
Oracle XML Database                       VALID      12.2.0.1.0  00:02:13
Final Actions                                                    00:02:34
Post Upgrade                                                     00:01:29
Post Compile                                                     00:06:22

Total Upgrade Time: 00:41:47 [PDB$SEED]

Database time zone version is 26. It meets current release needs.


Oracle Database 12.2 Post-Upgrade Status Tool           06-08-2017 12:27:52
[IRPDB]

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                          UPGRADED      12.2.0.1.0  00:27:49
Oracle Real Application Clusters       UPGRADED      12.2.0.1.0  00:00:00
Oracle Text                            UPGRADED      12.2.0.1.0  00:00:58
Oracle XML Database                    UPGRADED      12.2.0.1.0  00:02:13
Final Actions                                                    00:02:37
Post Upgrade                                                     00:01:30

Total Upgrade Time: 00:35:28 [IRPDB]

Database time zone version is 26. It meets current release needs.

15. Check registry history reflect the upgrade.

SQL> select * from cdb_registry_history;

ACTION_TIME                    ACTION          NAMESPACE  VERSION            ID COMMENTS                                                         CON_ID
------------------------------ --------------- ---------- ---------- ---------- ------------------------------------------------------------ ----------
                               BOOTSTRAP       DATAPATCH  12.1.0.2              RDBMS_12.1.0.2.0DBPSU_LINUX.X64_161210                                1
                               BOOTSTRAP       DATAPATCH  12.2.0.1              RDBMS_12.2.0.1.0_LINUX.X64_170125                                     1
08-JUN-17 04.12.06.632154 PM   VIEW INVALIDATE                          8289601 view invalidation                                                     1
08-JUN-17 04.14.00.598280 PM   UPGRADE         SERVER     12.2.0.1.0            Upgraded from 12.1.0.2.0                                              1
                               BOOTSTRAP       DATAPATCH  12.1.0.2              RDBMS_12.1.0.2.0DBPSU_LINUX.X64_161210                                3
                               BOOTSTRAP       DATAPATCH  12.2.0.1              RDBMS_12.2.0.1.0_LINUX.X64_170125                                     3
08-JUN-17 04.48.47.103524 PM   VIEW INVALIDATE                          8289601 view invalidation                                                     3
08-JUN-17 04.50.59.197968 PM   UPGRADE         SERVER     12.2.0.1.0            Upgraded from 12.1.0.2.0                                              3

SQL>  select con_id,comp_id,comp_name,version,status from cdb_registry order by 1;

    CON_ID COMP_ID    COMP_NAME                                VERSION    STATUS
---------- ---------- ---------------------------------------- ---------- ----------
         1 CATALOG    Oracle Database Catalog Views            12.2.0.1.0 VALID
         1 CATPROC    Oracle Database Packages and Types       12.2.0.1.0 VALID
         1 CONTEXT    Oracle Text                              12.2.0.1.0 VALID
         1 XDB        Oracle XML Database                      12.2.0.1.0 VALID
         1 RAC        Oracle Real Application Clusters         12.2.0.1.0 OPTION OFF
         3 CATALOG    Oracle Database Catalog Views            12.2.0.1.0 VALID
         3 CATPROC    Oracle Database Packages and Types       12.2.0.1.0 VALID
         3 CONTEXT    Oracle Text                              12.2.0.1.0 VALID
         3 XDB        Oracle XML Database                      12.2.0.1.0 VALID
         3 RAC        Oracle Real Application Clusters         12.2.0.1.0 OPTION OFF

16. Complete other post-upgrade steps and finally upgrade the compatible parameter.

ALTER SYSTEM SET compatible='12.2.0.1.0' SCOPE=SPFILE;

Useful metalink notes
Complete Checklist for Upgrading to Oracle Database 12c Release 2 (12.2) using DBUA [ID 2189854.1]
Complete Checklist for Manual Upgrades to Non-CDB Oracle Database 12c Release 2 (12.2) [ID 2173141.1]
Scripts to automatically update the RDBMS DST (timezone) version in an 11gR2 or 12c database . [ID 1585343.1]
Complete Checklist for upgrading Oracle 12.1 Container Database (CDB) to Oracle Database 12c Release 2 (12.2) using DBUA [ID 2240413.1]

Related Posts
Upgrading to 12.1.0.2 Using 11.2.0.4 Backups

↧

Upgrading to 12.2.0.1 Using 11.2.0.4 Backups

July 14, 2017, 12:00 am

≫ Next: getBoolean in 12c JDBC Driver and java.sql.SQLException: Fail to convert to internal representation

≪ Previous: Upgrading to 12.2.0.1 Using 12.1.0.2 Backups

This post list steps for upgrading a 11.2.0.4 single instance DB to 12.2.0.1 (non-CDB) using backups of the 11.2.0.4. There are two similar posts one for upgrading from 11.2.0.4 to 12.1.0.2 and another for upgrading from 12.1.0.2 to 12.2.0.1. It is assumed that a backup of the database is taken and copied to server where 12.2.0.1 software is installed.
1. Set the Oracle home to 12cR2 and oracle SID to the database to be restored.

echo $ORACLE_HOME
/opt/app/oracle/product/12.2.0/sehome_1

export ORACLE_SID=std11g4

2. Restore and recover the database

RMAN> startup nomount;
RMAN> create spfile from pfile;
RMAN> startup force nomount;

RMAN> restore controlfile from '/home/oracle/backup/control.bkp';
Starting restore at 13-JUN-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=191 device type=DISK

channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
output file name=/opt/app/oracle/oradata/STD11G4/controlfile/o1_mf_dmzw8m24_.ctl
output file name=/opt/app/oracle/fast_recovery/STD11G4/controlfile/o1_mf_dmzw8m6g_.ctl
Finished restore at 13-JUN-17

RMAN> alter database mount;
Statement processed
released channel: ORA_DISK_1

RMAN> crosscheck backupset;
Starting implicit crosscheck backup at 13-JUN-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=156 device type=DISK
Crosschecked 4 objects
Finished implicit crosscheck backup at 13-JUN-17

Starting implicit crosscheck copy at 13-JUN-17
using channel ORA_DISK_1
Finished implicit crosscheck copy at 13-JUN-17

searching for all files in the recovery area
cataloging files...
no files cataloged

using channel ORA_DISK_1
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/usr/local/backups/db0rs6mrr3_1_1.bkp RECID=25 STAMP=946564963
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/usr/local/backups/db0ss6mrto_1_1.bkp RECID=26 STAMP=946565048
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/usr/local/backups/db0ts6mrtp_1_1.bkp RECID=27 STAMP=946565049
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=/data/flash_recovery/STD11G4/autobackup/2017_06_13/o1_mf_s_946564859_dmzv1c4q_.bkp RECID=28 STAMP=946565051
Crosschecked 4 objects


RMAN> delete noprompt expired backup;
using channel ORA_DISK_1
deleted backup piece
backup piece handle=/usr/local/backups/db0rs6mrr3_1_1.bkp RECID=25 STAMP=946564963
deleted backup piece
backup piece handle=/usr/local/backups/db0ss6mrto_1_1.bkp RECID=26 STAMP=946565048
deleted backup piece
backup piece handle=/usr/local/backups/db0ts6mrtp_1_1.bkp RECID=27 STAMP=946565049
deleted backup piece
backup piece handle=/data/flash_recovery/STD11G4/autobackup/2017_06_13/o1_mf_s_946564859_dmzv1c4q_.bkp RECID=28 STAMP=946565051
Deleted 4 EXPIRED objects


RMAN> catalog  start with '/home/oracle/backup/' noprompt;
searching for all files that match the pattern /home/oracle/backup/

List of Files Unknown to the Database
=====================================
File Name: /home/oracle/backup/res.rmn
File Name: /home/oracle/backup/control.bkp
File Name: /home/oracle/backup/db0rs6mrr3_1_1.bkp
File Name: /home/oracle/backup/db0ss6mrto_1_1.bkp
File Name: /home/oracle/backup/db0ts6mrtp_1_1.bkp
File Name: /home/oracle/backup/pfile.ora
cataloging files...
cataloging done

List of Cataloged Files
=======================
File Name: /home/oracle/backup/control.bkp
File Name: /home/oracle/backup/db0rs6mrr3_1_1.bkp
File Name: /home/oracle/backup/db0ss6mrto_1_1.bkp
File Name: /home/oracle/backup/db0ts6mrtp_1_1.bkp

List of Files Which Were Not Cataloged
=======================================
File Name: /home/oracle/backup/res.rmn
  RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/backup/pfile.ora
  RMAN-07517: Reason: The file header is corrupted

RMAN> run {
2> restore database;
3> recover database;
4> }
Starting restore at 13-JUN-17
using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /data/oradata/STD11G4/datafile/o1_mf_system_c783t3v8_.dbf
channel ORA_DISK_1: restoring datafile 00002 to /data/oradata/STD11G4/datafile/o1_mf_sysaux_c783t3vy_.dbf
channel ORA_DISK_1: restoring datafile 00003 to /data/oradata/STD11G4/datafile/o1_mf_undotbs1_c783t3x3_.dbf
channel ORA_DISK_1: restoring datafile 00004 to /data/oradata/STD11G4/datafile/o1_mf_users_c783t3ss_.dbf
channel ORA_DISK_1: restoring datafile 00005 to /data/oradata/STD11G4/datafile/o1_mf_tools_c783t3rr_.dbf
channel ORA_DISK_1: restoring datafile 00006 to /data/oradata/STD11G4/datafile/o1_mf_abc_c783t3yz_.dbf
channel ORA_DISK_1: restoring datafile 00009 to /data/oradata/STD11G4/datafile/o1_mf_abc_crdch1bd_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/db0rs6mrr3_1_1.bkp
channel ORA_DISK_1: piece handle=/home/oracle/backup/db0rs6mrr3_1_1.bkp tag=TAG20170613T144243
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:02:16
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00007 to /data/oradata/STD11G4/datafile/o1_mf_tbs2k_cwto1wx1_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/db0ss6mrto_1_1.bkp
channel ORA_DISK_1: piece handle=/home/oracle/backup/db0ss6mrto_1_1.bkp tag=TAG20170613T144243
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00008 to /data/oradata/STD11G4/datafile/o1_mf_tbs32k_cwto2cr8_.dbf
channel ORA_DISK_1: reading from backup piece /home/oracle/backup/db0ts6mrtp_1_1.bkp
channel ORA_DISK_1: piece handle=/home/oracle/backup/db0ts6mrtp_1_1.bkp tag=TAG20170613T144243
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 13-JUN-17

Starting recover at 13-JUN-17
using channel ORA_DISK_1

starting media recovery

RMAN-08187: warning: media recovery until SCN 13940762 complete
Finished recover at 13-JUN-17

3. Instead of the "open resetlogs" open the database using "open resetlogs upgrade". The errors are ignorable.

RMAN> alter database open resetlogs upgrade;
Statement processed
RMAN-06900: warning: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT row
RMAN-06901: warning: disabling update of the V$RMAN_STATUS and V$RMAN_OUTPUT rows
ORACLE error from target database:
ORA-04023: Object SYS.STANDARD could not be validated or authorized

4. Check the database status, it should be in open migrate status.

SQL> select status from v$instance;

STATUS
------------
OPEN MIGRATE

5. Run the dbupgrade to commence the DB upgrade to 12.2

$ dbupgrade

Argument list for [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catctl.pl]
Run in                c = 0
Do not run in         C = 0
Input Directory       d = 0
Echo OFF              e = 1
Simulate              E = 0
Forced cleanup        F = 0
Log Id                i = 0
Child Process         I = 0
Log Dir               l = 0
Priority List Name    L = 0
Upgrade Mode active   M = 0
SQL Process Count     n = 0
SQL PDB Process Count N = 0
Open Mode Normal      o = 0
Start Phase           p = 0
End Phase             P = 0
Reverse Order         r = 0
AutoUpgrade Resume    R = 0
Script                s = 0
Serial Run            S = 0
RO User Tablespaces   T = 0
Display Phases        y = 0
Debug catcon.pm       z = 0
Debug catctl.pl       Z = 0

catctl.pl VERSION: [12.2.0.1.0]
           STATUS: [production]
            BUILD: [RDBMS_12.2.0.1.0_LINUX.X64_170125]


/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/orahome = [/opt/app/oracle/product/12.2.0/sehome_1]
/opt/app/oracle/product/12.2.0/sehome_1/bin/orabasehome = [/opt/app/oracle/product/12.2.0/sehome_1]
catctlGetOrabase = [/opt/app/oracle/product/12.2.0/sehome_1]

Analyzing file /opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/catupgrd.sql

Log file directory = [/tmp/cfgtoollogs/upgrade20170613150844]

catcon: ALL catcon-related output will be written to [/tmp/cfgtoollogs/upgrade20170613150844/catupgrd_catcon_14703.lst]
catcon: See [/tmp/cfgtoollogs/upgrade20170613150844/catupgrd*.log] files for output generated by scripts
catcon: See [/tmp/cfgtoollogs/upgrade20170613150844/catupgrd_*.lst] files for spool files, if any

Number of Cpus        = 8
Database Name         = STD11G4
DataBase Version      = 11.2.0.4.0
catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846/catupgrd_catcon_14703.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846/catupgrd*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846/catupgrd_*.lst] files for spool files, if any

Log file directory = [/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846]

Parallel SQL Process Count            = 4
Components in [STD11G4]
    Installed [APS CATALOG CATJAVA CATPROC CONTEXT EM JAVAVM OWM XDB XML XOQ]
Not Installed [APEX DV MGW ODM OLS ORDIM RAC SDO WK]

------------------------------------------------------
Phases [0-115]         Start Time:[2017_06_13 15:08:46]
------------------------------------------------------
***********   Executing Change Scripts   ***********
Serial   Phase #:0    [STD11G4] Files:1    Time: 177s
***************   Catalog Core SQL   ***************
Serial   Phase #:1    [STD11G4] Files:5    Time: 126s
Restart  Phase #:2    [STD11G4] Files:1    Time: 1s
***********   Catalog Tables and Views   ***********
Parallel Phase #:3    [STD11G4] Files:19   Time: 48s
Restart  Phase #:4    [STD11G4] Files:1    Time: 0s
*************   Catalog Final Scripts   ************
Serial   Phase #:5    [STD11G4] Files:6    Time: 43s
*****************   Catproc Start   ****************
Serial   Phase #:6    [STD11G4] Files:1    Time: 28s
*****************   Catproc Types   ****************
Serial   Phase #:7    [STD11G4] Files:2    Time: 25s
Restart  Phase #:8    [STD11G4] Files:1    Time: 0s
****************   Catproc Tables   ****************
Parallel Phase #:9    [STD11G4] Files:69   Time: 59s
Restart  Phase #:10   [STD11G4] Files:1    Time: 1s
*************   Catproc Package Specs   ************
Serial   Phase #:11   [STD11G4] Files:1    Time: 97s
Restart  Phase #:12   [STD11G4] Files:1    Time: 0s
**************   Catproc Procedures   **************
Parallel Phase #:13   [STD11G4] Files:97   Time: 16s
Restart  Phase #:14   [STD11G4] Files:1    Time: 0s
Parallel Phase #:15   [STD11G4] Files:118  Time: 25s
Restart  Phase #:16   [STD11G4] Files:1    Time: 1s
Serial   Phase #:17   [STD11G4] Files:13   Time: 5s
Restart  Phase #:18   [STD11G4] Files:1    Time: 0s
*****************   Catproc Views   ****************
Parallel Phase #:19   [STD11G4] Files:33   Time: 41s
Restart  Phase #:20   [STD11G4] Files:1    Time: 0s
Serial   Phase #:21   [STD11G4] Files:3    Time: 15s
Restart  Phase #:22   [STD11G4] Files:1    Time: 0s
Parallel Phase #:23   [STD11G4] Files:24   Time: 106s
Restart  Phase #:24   [STD11G4] Files:1    Time: 0s
Parallel Phase #:25   [STD11G4] Files:11   Time: 63s
Restart  Phase #:26   [STD11G4] Files:1    Time: 0s
Serial   Phase #:27   [STD11G4] Files:1    Time: 0s
Serial   Phase #:28   [STD11G4] Files:3    Time: 6s
Serial   Phase #:29   [STD11G4] Files:1    Time: 0s
Restart  Phase #:30   [STD11G4] Files:1    Time: 0s
***************   Catproc CDB Views   **************
Serial   Phase #:31   [STD11G4] Files:1    Time: 0s
Restart  Phase #:32   [STD11G4] Files:1    Time: 1s
Serial   Phase #:34   [STD11G4] Files:1    Time: 0s
*****************   Catproc PLBs   *****************
Serial   Phase #:35   [STD11G4] Files:283  Time: 35s
Serial   Phase #:36   [STD11G4] Files:1    Time: 0s
Restart  Phase #:37   [STD11G4] Files:1    Time: 0s
Serial   Phase #:38   [STD11G4] Files:1    Time: 7s
Restart  Phase #:39   [STD11G4] Files:1    Time: 0s
***************   Catproc DataPump   ***************
Serial   Phase #:40   [STD11G4] Files:3    Time: 70s
Restart  Phase #:41   [STD11G4] Files:1    Time: 0s
******************   Catproc SQL   *****************
Parallel Phase #:42   [STD11G4] Files:13   Time: 63s
Restart  Phase #:43   [STD11G4] Files:1    Time: 1s
Parallel Phase #:44   [STD11G4] Files:12   Time: 14s
Restart  Phase #:45   [STD11G4] Files:1    Time: 0s
Parallel Phase #:46   [STD11G4] Files:2    Time: 1s
Restart  Phase #:47   [STD11G4] Files:1    Time: 0s
*************   Final Catproc scripts   ************
Serial   Phase #:48   [STD11G4] Files:1    Time: 13s
Restart  Phase #:49   [STD11G4] Files:1    Time: 0s
**************   Final RDBMS scripts   *************
Serial   Phase #:50   [STD11G4] Files:1    Time: 39s
************   Upgrade Component Start   ***********
Serial   Phase #:51   [STD11G4] Files:1    Time: 0s
Restart  Phase #:52   [STD11G4] Files:1    Time: 1s
****************   Upgrading Java   ****************
Serial   Phase #:53   [STD11G4] Files:1    Time: 0s
Restart  Phase #:54   [STD11G4] Files:1    Time: 0s
*****************   Upgrading XDK   ****************
Serial   Phase #:55   [STD11G4] Files:1    Time: 0s
Restart  Phase #:56   [STD11G4] Files:1    Time: 0s
*********   Upgrading APS,OLS,DV,CONTEXT   *********
Serial   Phase #:57   [STD11G4] Files:1    Time: 77s
*****************   Upgrading XDB   ****************
Restart  Phase #:58   [STD11G4] Files:1    Time: 0s
Serial   Phase #:60   [STD11G4] Files:3    Time: 21s
Serial   Phase #:61   [STD11G4] Files:3    Time: 9s
Parallel Phase #:62   [STD11G4] Files:9    Time: 4s
Parallel Phase #:63   [STD11G4] Files:24   Time: 4s
Serial   Phase #:64   [STD11G4] Files:4    Time: 13s
Serial   Phase #:65   [STD11G4] Files:1    Time: 0s
Serial   Phase #:66   [STD11G4] Files:30   Time: 5s
Serial   Phase #:67   [STD11G4] Files:1    Time: 0s
Parallel Phase #:68   [STD11G4] Files:6    Time: 3s
Serial   Phase #:69   [STD11G4] Files:2    Time: 22s
Serial   Phase #:70   [STD11G4] Files:3    Time: 68s
Restart  Phase #:71   [STD11G4] Files:1    Time: 0s
*********   Upgrading CATJAVA,OWM,MGW,RAC   ********
Serial   Phase #:72   [STD11G4] Files:1    Time: 80s
****************   Upgrading ORDIM   ***************
Restart  Phase #:73   [STD11G4] Files:1    Time: 0s
Serial   Phase #:75   [STD11G4] Files:1    Time: 0s
Parallel Phase #:76   [STD11G4] Files:2    Time: 0s
Serial   Phase #:77   [STD11G4] Files:1    Time: 1s
Restart  Phase #:78   [STD11G4] Files:1    Time: 0s
Parallel Phase #:79   [STD11G4] Files:2    Time: 0s
Serial   Phase #:80   [STD11G4] Files:2    Time: 0s
*****************   Upgrading SDO   ****************
Restart  Phase #:81   [STD11G4] Files:1    Time: 1s
Serial   Phase #:83   [STD11G4] Files:1    Time: 0s
Serial   Phase #:84   [STD11G4] Files:1    Time: 0s
Restart  Phase #:85   [STD11G4] Files:1    Time: 0s
Serial   Phase #:86   [STD11G4] Files:1    Time: 1s
Restart  Phase #:87   [STD11G4] Files:1    Time: 0s
Parallel Phase #:88   [STD11G4] Files:3    Time: 0s
Restart  Phase #:89   [STD11G4] Files:1    Time: 0s
Serial   Phase #:90   [STD11G4] Files:1    Time: 0s
Restart  Phase #:91   [STD11G4] Files:1    Time: 1s
Serial   Phase #:92   [STD11G4] Files:1    Time: 0s
Restart  Phase #:93   [STD11G4] Files:1    Time: 0s
Parallel Phase #:94   [STD11G4] Files:4    Time: 0s
Restart  Phase #:95   [STD11G4] Files:1    Time: 0s
Serial   Phase #:96   [STD11G4] Files:1    Time: 1s
Restart  Phase #:97   [STD11G4] Files:1    Time: 0s
Serial   Phase #:98   [STD11G4] Files:2    Time: 0s
Restart  Phase #:99   [STD11G4] Files:1    Time: 0s
Serial   Phase #:100  [STD11G4] Files:1    Time: 1s
Restart  Phase #:101  [STD11G4] Files:1    Time: 0s
***********   Upgrading Misc. ODM, OLAP   **********
Serial   Phase #:102  [STD11G4] Files:1    Time: 0s
****************   Upgrading APEX   ****************
Restart  Phase #:103  [STD11G4] Files:1    Time: 0s
Serial   Phase #:104  [STD11G4] Files:1    Time: 0s
Restart  Phase #:105  [STD11G4] Files:1    Time: 1s
***********   Final Component scripts    ***********
Serial   Phase #:106  [STD11G4] Files:1    Time: 0s
*************   Final Upgrade scripts   ************
Serial   Phase #:107  [STD11G4] Files:1    Time: 81s
**********   End PDB Application Upgrade   *********
Serial   Phase #:108  [STD11G4] Files:1    Time: 0s
*******************   Migration   ******************
Serial   Phase #:109  [STD11G4] Files:1    Time: 46s
Serial   Phase #:110  [STD11G4] Files:1    Time: 0s
Serial   Phase #:111  [STD11G4] Files:1    Time: 48s
*****************   Post Upgrade   *****************
Serial   Phase #:112  [STD11G4] Files:1    Time: 252s
****************   Summary report   ****************
Serial   Phase #:113  [STD11G4] Files:1    Time: 0s
Serial   Phase #:114  [STD11G4] Files:1    Time: 0s
Serial   Phase #:115  [STD11G4] Files:1     Time: 29s

------------------------------------------------------
Phases [0-115]         End Time:[2017_06_13 15:40:23]
------------------------------------------------------

Grand Total Time: 1899s

 LOG FILES: (/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846/catupgrd*.log)

Upgrade Summary Report Located in:
/opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846/upg_summary.log

Grand Total Upgrade Time:    [0d:0h:31m:39s]

6. View the upgrade summary. This shows that time-zone file need upgrading to latest version.

$ cat /opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846/upg_summary.log

Oracle Database 12.2 Post-Upgrade Status Tool           06-13-2017 15:39:54

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                          UPGRADED      12.2.0.1.0  00:18:41
Oracle Workspace Manager               UPGRADED      12.2.0.1.0  00:01:19
OLAP Analytic Workspace              OPTION OFF      11.2.0.2.0  00:00:00
Oracle OLAP API                      OPTION OFF      11.2.0.2.0  00:00:00
Oracle Text                            UPGRADED      12.2.0.1.0  00:01:15
Oracle XML Database                    UPGRADED      12.2.0.1.0  00:02:28
Final Actions                                                    00:02:06
Post Upgrade                                                     00:04:11

Total Upgrade Time: 00:30:13

Database time zone version is 14. It is older than current release time
zone version 26. Time zone upgrade is needed using the DBMS_DST package.

Grand Total Upgrade Time:    [0d:0h:31m:39s]

7. Before the timezone upgrade run the utlrp to compile packages.

$ORACLE_HOME/perl/bin/perl catcon.pl -n 1 -e -b utlrp -d '''.''' utlrp.sql
catcon: ALL catcon-related output will be written to [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/utlrp_catcon_24388.lst]
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/utlrp*.log] files for output generated by scripts
catcon: See [/opt/app/oracle/product/12.2.0/sehome_1/rdbms/admin/utlrp_*.lst] files for spool files, if any
catcon.pl: completed successfully

8. Check the timezone upgrade state on the database. Initially the upgrade state will be 0 and secondary timezone will be 0. Refer here for further information on timezone upgrade.

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME; 

PROPERTY_NAME             VALUE
------------------------- ----------
DST_PRIMARY_TT_VERSION    14
DST_SECONDARY_TT_VERSION  0
DST_UPGRADE_STATE         NONE

9. Start the database in upgrade mode and being the timezone upgrade.

SQL> shutdown immediate;
SQL> startup upgrade;


exec DBMS_DST.BEGIN_UPGRADE(26);

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME;   

PROPERTY_NAME             VALUE
------------------------- ----------
DST_PRIMARY_TT_VERSION    26
DST_SECONDARY_TT_VERSION  14
DST_UPGRADE_STATE         UPGRADE

10. Restar the database in normal mode and complete the timezone upgrade.

shutdown immedaite;
startup;

truncate table sys.dst$error_table;
truncate table sys.dst$trigger_table;

VAR numfail number
BEGIN
DBMS_DST.UPGRADE_DATABASE(:numfail,
parallel                  => TRUE,
log_errors                => TRUE,
log_errors_table          =>'SYS.DST$ERROR_TABLE',
log_triggers_table        =>'SYS.DST$TRIGGER_TABLE',
error_on_overlap_time     => TRUE,
error_on_nonexisting_time => TRUE);
DBMS_OUTPUT.PUT_LINE('Failures:'|| :numfail);
END;
/
exec DBMS_DST.END_UPGRADE(:numfail);

11. Verify timezone has been upgraded to 26.

SQL> select * from v$timezone_file;

FILENAME                VERSION     CON_ID
-------------------- ---------- ----------
timezlrg_26.dat              26          0

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME;  

PROPERTY_NAME             VALUE
------------------------- ----------
DST_PRIMARY_TT_VERSION    26
DST_SECONDARY_TT_VERSION  0
DST_UPGRADE_STATE         NONE


SQL> @?/rdbms/admin/utlu122s.sql

Oracle Database 12.2 Post-Upgrade Status Tool           06-13-2017 15:50:24

Component                               Current         Version  Elapsed Time
Name                                    Status          Number   HH:MM:SS

Oracle Server                             VALID      12.2.0.1.0  00:18:41
Oracle Workspace Manager                  VALID      12.2.0.1.0  00:01:19
OLAP Analytic Workspace              OPTION OFF      11.2.0.2.0  00:00:00
Oracle OLAP API                      OPTION OFF      11.2.0.2.0  00:00:00
Oracle Text                               VALID      12.2.0.1.0  00:01:15
Oracle XML Database                       VALID      12.2.0.1.0  00:02:28
Final Actions                                                    00:02:06
Post Upgrade                                                     00:04:11
Post Compile                                                     00:01:52

Total Upgrade Time: 00:32:05

Database time zone version is 26. It meets current release needs.

Summary Report File = /opt/app/oracle/product/12.2.0/sehome_1/cfgtoollogs/STD11G4/upgrade20170613150846/upg_summary.log

12. As part of the upgrade statspack was upgraded to 12.2 (refer 394937.1).

SQL> @?/rdbms/admin/spup112

Statspack Upgrade script
~~~~~~~~~~~~~~~~~~~~~~~~

Warning
~~~~~~~
Converting existing Statspack data to 12.1 format may result in
irregularities when reporting on pre-12.1 snapshot data.

This script is provided for convenience, and is not guaranteed to
work on all installations.  To ensure you will not lose any existing
Statspack data, export the schema before upgrading.  A downgrade
script is not provided.  Please see spdoc.txt for more details.

Press return before continuing

Usage
~~~~~
-> Disable any programs which run Statspack (including any dbms_jobs),
before continuing, or this upgrade will fail.

-> You MUST be connected as a user with SYSDBA privilege to successfully
run this script.

-> You will be prompted for the PERFSTAT password, and for the
tablespace to create any new PERFSTAT tables/indexes.

Press return before continuing

Please specify the PERFSTAT password
Enter value for perfstat_password: test
test
Note:
Please check remainder of upgrade log file, which is continued in
the file spup112b.lis

Grant succeeded.

Connected.
USER is "PERFSTAT"
insert into STATS$IDLE_EVENT (event) values ('GCR sleep')
*
ERROR at line 1:
ORA-00001: unique constraint (PERFSTAT.STATS$IDLE_EVENT_PK) violated

insert into STATS$IDLE_EVENT (event) values ('LogMiner builder: branch')
*
ERROR at line 1:
ORA-00001: unique constraint (PERFSTAT.STATS$IDLE_EVENT_PK) violated

insert into STATS$IDLE_EVENT (event) values ('LogMiner builder: idle')
*
ERROR at line 1:
ORA-00001: unique constraint (PERFSTAT.STATS$IDLE_EVENT_PK) violated

insert into STATS$IDLE_EVENT (event) values ('LogMiner client: transaction')
*
ERROR at line 1:
ORA-00001: unique constraint (PERFSTAT.STATS$IDLE_EVENT_PK) violated

insert into STATS$IDLE_EVENT (event) values ('LogMiner preparer: idle')
*
ERROR at line 1:
ORA-00001: unique constraint (PERFSTAT.STATS$IDLE_EVENT_PK) violated

insert into STATS$IDLE_EVENT (event) values ('parallel recovery control message reply')
*
ERROR at line 1:
ORA-00001: unique constraint (PERFSTAT.STATS$IDLE_EVENT_PK) violated

1 row created.
...
1 row created.

Commit complete.
Table altered.
Commit complete.
Table created.
Synonym created.
Commit complete.

Note:
Please check the log file of the package recreation, which is
in the file spcpkg.lis
Creating Package STATSPACK...

Package created.

No errors.
Creating Package Body STATSPACK...

Warning: Package Body created with compilation errors.

Errors for PACKAGE BODY STATSPACK:

LINE/COL ERROR
-------- -----------------------------------------------------------------
3219/8   PL/SQL: SQL Statement ignored
3223/15  PL/SQL: ORA-00904: "REMASTER_TYPE": invalid identifier

NOTE:
SPCPKG complete. Please check spcpkg.lis for any errors.

SQL>
SQL> --  End of Upgrade script

conn / as sysdba

SQL>@?/rdbms/admin/spup12200.sql

13. Finally update the compatibility parameter to 12.2

SQL> alter system set compatible='12.2.0.1.0' scope=spfile;

14. This concludes the upgrade to 12.2.

SQL> select * from dba_registry_history order by 1;

ACTION_TIME                    ACTION          NAMESPACE                      VERSION                                ID COMMENTS
------------------------------ --------------- ------------------------------ ------------------------------ ---------- ----------------------------------------
05-SEP-10 06.22.14.370943 AM   APPLY           SERVER                         11.2.0.2                                0 Patchset 11.2.0.2.0
28-JAN-11 02.04.20.929972 PM   APPLY           SERVER                         11.2.0.2                                1 PSU 11.2.0.2.1
28-JAN-11 02.08.47.891548 PM   APPLY           SERVER                         11.2.0.2                                1 PSU 11.2.0.2.1
24-MAY-11 01.32.49.622817 PM   APPLY           SERVER                         11.2.0.2                                2 PSU 11.2.0.2.2
20-JUL-11 11.46.58.623557 AM   APPLY           SERVER                         11.2.0.2                                3 PSU 11.2.0.2.3
19-OCT-11 11.56.37.705897 AM   APPLY           SERVER                         11.2.0.2                                4 PSU 11.2.0.2.4
18-JAN-12 03.25.41.512830 PM   APPLY           SERVER                         11.2.0.2                                5 PSU 11.2.0.2.5
13-FEB-12 03.27.42.343779 PM   VIEW INVALIDATE                                                                  8289601 view invalidation
13-FEB-12 03.27.42.835430 PM   UPGRADE         SERVER                         11.2.0.3.0                                Upgraded from 11.2.0.2.0
13-FEB-12 03.28.03.564686 PM   APPLY           SERVER                         11.2.0.3                                0 Patchset 11.2.0.2.0
30-APR-12 12.23.17.690100 PM   APPLY           SERVER                         11.2.0.3                                2 PSU 11.2.0.3.2
18-JUL-12 10.48.29.139682 AM   APPLY           SERVER                         11.2.0.3                                3 PSU 11.2.0.3.3
06-NOV-12 02.33.09.598897 PM   APPLY           SERVER                         11.2.0.3                                4 PSU 11.2.0.3.4
21-JAN-13 10.59.24.458354 AM   APPLY           SERVER                         11.2.0.3                                5 PSU 11.2.0.3.5
19-APR-13 04.27.41.823099 PM   APPLY           SERVER                         11.2.0.3                                6 PSU 11.2.0.3.6
29-JUL-13 02.28.13.060031 PM   APPLY           SERVER                         11.2.0.3                                7 PSU 11.2.0.3.7
22-OCT-13 05.17.58.670146 PM   APPLY           SERVER                         11.2.0.3                                8 PSU 11.2.0.3.8
21-JAN-14 10.40.22.294040 AM   APPLY           SERVER                         11.2.0.3                                9 PSU 11.2.0.3.9
30-APR-14 03.58.10.311583 PM   APPLY           SERVER                         11.2.0.3                               10 PSU 11.2.0.3.10
16-JUL-14 01.25.56.344898 PM   APPLY           SERVER                         11.2.0.3                               11 PSU 11.2.0.3.11
11-NOV-14 02.59.11.471061 PM   APPLY           SERVER                         11.2.0.3                               12 PSU 11.2.0.3.12
21-JAN-15 11.47.41.906220 AM   jvmpsu.sql      SERVER                         11.2.0.3.2OJVMBP                        0 RAN jvmpsu.sql
21-JAN-15 11.47.41.926803 AM   APPLY           SERVER                         11.2.0.3.2OJVMBP                        0 OJVM PSU post-install
21-JAN-15 11.50.26.504554 AM   APPLY           SERVER                         11.2.0.3                               13 PSU 11.2.0.3.13
18-DEC-15 02.29.09.949010 PM   VIEW INVALIDATE                                                                  8289601 view invalidation
18-DEC-15 02.29.09.994073 PM   UPGRADE         SERVER                         11.2.0.4.0                                Upgraded from 11.2.0.3.0
18-DEC-15 02.29.31.582833 PM   APPLY           SERVER                         11.2.0.4                                0 Patchset 11.2.0.2.0
18-DEC-15 02.47.17.164443 PM   APPLY           SERVER                         11.2.0.4                                8 PSU 11.2.0.4.8
20-APR-16 01.39.00.784851 PM   APPLY           SERVER                         11.2.0.4                           160419 PSU 11.2.0.4.160419
13-JUN-17 03.33.00.073857 PM   VIEW INVALIDATE                                                                  8289601 view invalidation
13-JUN-17 03.34.54.163714 PM   UPGRADE         SERVER                         12.2.0.1.0                                Upgraded from 11.2.0.4.0
                               BOOTSTRAP       DATAPATCH                      12.2.0.1                                  RDBMS_12.2.0.1.0_LINUX.X64_170125


SQL> select comp_id,status,version from dba_registry order  by 2,3;

COMP_ID                        STATUS                                       VERSION
------------------------------ -------------------------------------------- -----------
APS                            OPTION OFF                                   11.2.0.2.0
XOQ                            OPTION OFF                                   11.2.0.2.0
XML                            REMOVED                                      11.2.0.3.0
JAVAVM                         REMOVED                                      11.2.0.3.0
CATJAVA                        REMOVED                                      11.2.0.3.0
CATPROC                        VALID                                        12.2.0.1.0
CATALOG                        VALID                                        12.2.0.1.0
OWM                            VALID                                        12.2.0.1.0
CONTEXT                        VALID                                        12.2.0.1.0
XDB                            VALID                                        12.2.0.1.0

Related Posts
Upgrading to 12.1.0.2 Using 11.2.0.4 Backups
Upgrading to 12.2.0.1 Using 12.1.0.2 Backups

↧