Upgrading 11.2.0.3 (11gR2) Database to 12.1.0.1 (12c) Using DBUA

After upgrading grid infrastructure and RAC software next step is to upgrade the actual database. This could be done manually or using DBUA. There are several metalink notes (listed at the end of the post) that list comprehensive set of instructions for database upgrades. It's advisable to go through them before upgrading any production system, this post only list highlights and few issues encountered along the way.
Although the 12c installation has the utlu121i.sql script it's been depreciated. Running this would give the following output listing the correct script to run.

SQL> @utlu121i.sql
DOC>############################################################################
DOC>############################################################################
DOC>
DOC>
DOC>Note that the new preupgrd.sql script replaces the utlu121i.sql script
DOC>  and earlier versions of the Pre-Upgrade Information Tool.
DOC>
DOC>  The Pre-Upgrade Information Tool, which now consists of preupgrd.sql
DOC>  and utluppkg.sql.
DOC>
DOC>  1. Make sure preupgrd.sql and utluppkg.sql (located in the admin
DOC>     directory of the new installation of Oracle Database 12.1) are
DOC>     accessible while connected to your source database, which is the
DOC>     database to be upgraded
DOC>
DOC>  2. Connect to your source database using an account with DBA privileges
DOC>      and execute the preupgrd.sql script.
DOC>
DOC>  For more information on the Pre-Upgrade Information Tool, see the
DOC>  Oracle Database Upgrade Guide.
DOC>
DOC>
DOC>############################################################################
DOC>############################################################################
DOC>#

Running the new preupgrd.sql will generate three files in $ORACLE_BASE/cfgtoollogs/DB_NAME/preupgrade directory.

@preupgrd.sql
Results of the checks are located at:
 /opt/app/oracle/cfgtoollogs/std11g2/preupgrade/preupgrade.log

Pre-Upgrade Fixup Script (run in source database environment):
 /opt/app/oracle/cfgtoollogs/std11g2/preupgrade/preupgrade_fixups.sql

Post-Upgrade Fixup Script (run shortly after upgrade):
 /opt/app/oracle/cfgtoollogs/std11g2/preupgrade/postupgrade_fixups.sql

The log file will have the recommendation and warnings.

cat preupgrade.log

      ************************************************************

         Fixup scripts must be reviewed prior to being executed.

      ************************************************************

      ************************************************************
                   ====>> USER ACTION REQUIRED  <<====
      ************************************************************

 The following are *** ERROR LEVEL CONDITIONS *** that must be addressed
                    prior to attempting your upgrade.
            Failure to do so will result in a failed upgrade.

           You MUST resolve the above errors prior to upgrade

      ************************************************************



[oracle@rhel6m1 admin]$ cat /opt/app/oracle/cfgtoollogs/std11g2/preupgrade/preupgrade.log
Oracle Database Pre-Upgrade Information Tool 07-16-2013 11:25:22
Script Version: 12.1.0.1.0 Build: 006
**********************************************************************
   Database Name:  STD11G2
         Version:  11.2.0.3.0
      Compatible:  11.2.0.0.0
       Blocksize:  8192
        Platform:  Linux x86 64-bit
   Timezone file:  V14
**********************************************************************
                          [Renamed Parameters]
                     [No Renamed Parameters in use]
**********************************************************************
**********************************************************************
                    [Obsolete/Deprecated Parameters]
             [No Obsolete or Desupported Parameters in use]
**********************************************************************
                            [Component List]
**********************************************************************
--> Oracle Catalog Views                   [upgrade]  VALID
--> Oracle Packages and Types              [upgrade]  VALID
--> JServer JAVA Virtual Machine           [upgrade]  VALID
--> Oracle XDK for Java                    [upgrade]  VALID
--> Real Application Clusters              [upgrade]  VALID
--> Oracle Workspace Manager               [upgrade]  VALID
--> Oracle XML Database                    [upgrade]  VALID
--> Oracle Java Packages                   [upgrade]  VALID
--> Oracle Ultra Search                    [upgrade]
--> Expression Filter                      [upgrade]  VALID
--> Rule Manager                           [upgrade]  VALID
**********************************************************************
                              [Tablespaces]
**********************************************************************
--> SYSTEM tablespace is adequate for the upgrade.
     minimum required size: 944 MB
--> SYSAUX tablespace is adequate for the upgrade.
     minimum required size: 572 MB
--> UNDOTBS1 tablespace is adequate for the upgrade.
     minimum required size: 400 MB
--> TEMP tablespace is adequate for the upgrade.
     minimum required size: 60 MB

                      [No adjustments recommended]

**********************************************************************
**********************************************************************
                          [Pre-Upgrade Checks]
**********************************************************************
INFORMATION: --> Older Timezone in use

     Database is using a time zone file older than version 18.
     After the upgrade, it is recommended that DBMS_DST package
     be used to upgrade the 11.2.0.3.0 database time zone version
     to the latest version which comes with the new release.
     Please refer to My Oracle Support note number 977512.1 for details.


**********************************************************************
                      [Pre-Upgrade Recommendations]
**********************************************************************

                        *****************************************
                        ********* Dictionary Statistics *********
                        *****************************************

Please gather dictionary statistics 24 hours prior to
upgrading the database.
To gather dictionary statistics execute the following command
while connected as SYSDBA:
    EXECUTE dbms_stats.gather_dictionary_stats;

^^^ MANUAL ACTION SUGGESTED ^^^

**********************************************************************
                     [Post-Upgrade Recommendations]
**********************************************************************

                        *****************************************
                        ******** Fixed Object Statistics ********
                        *****************************************

Please create stats on fixed objects two weeks
after the upgrade using the command:
   EXECUTE DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;

^^^ MANUAL ACTION SUGGESTED ^^^

**********************************************************************
                   ************  Summary  ************

 0 ERRORS exist in your database.
 0 WARNINGS exist in your database.
 1 INFORMATIONAL message that should be reviewed prior to your upgrade.

 After your database is upgraded and open in normal mode you must run
 rdbms/admin/catuppst.sql which executes several required tasks and completes
 the upgrade process.

 You should follow that with the execution of rdbms/admin/utlrp.sql, and a
 comparison of invalid objects before and after the upgrade using
 rdbms/admin/utluiobj.sql

 If needed you may want to upgrade your timezone data using the process
 described in My Oracle Support note 977512.1
                   ***********************************

Run the pre-upgrade script to fix any errors and warnings.
Timezone upgrade is another thing that must be considered during the upgrade. When upgrading from 11.2.0.3 and if current time zone is less than 18 then timezone must be upgraded after the database is upgraded (or could be done using DBUA). Metalink notes 1509653.1 and 1522719.1 has useful information for this upgrade path. Current timezone settings are

SQL> SELECT version FROM v$timezone_file;
        14

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
  2  FROM DATABASE_PROPERTIES
  3  WHERE PROPERTY_NAME LIKE 'DST_%'
  4  ORDER BY PROPERTY_NAME;
DST_PRIMARY_TT_VERSION         14
DST_SECONDARY_TT_VERSION       0
DST_UPGRADE_STATE              NONE

As mentioned in the grid infrastructure upgrade in this environment the listener name has been changed. But the DBUA still looks for listener with default name and without it cannot proceed any further (ignoring this didn't help either).
To fix this add a listener with default name "LISTENER" and start it

[grid@rhel6m1 admin]$ srvctl add listener -listener listener -p 1521
[grid@rhel6m1 admin]$ srvctl start listener -l listener
[grid@rhel6m1 admin]$ lsnrctl status

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 12.1.0.1.0 - Production
Start Date                16-JUL-2013 11:46:56
..
Listener Parameter File   /opt/app/12.1.0/grid/network/admin/listener.ora
Listener Log File         /opt/app/oracle/diag/tnslsnr/rhel6m1/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.93)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.97)(PORT=1521)))
The listener supports no services
The command completed successfully

The database is standard edition database and there's no Oracle text component on it (see registry components above). However during the upgrade following warning occurs.
It was possible to ignore this warning and complete the upgrade. Oracle confirmed this when an SR was raised saying "Please proceed with the upgrade ignoring the ctxsys.ctx_adm must be declared warning as Oracle Text is not being used."
Another issue encountered was spfile being referred directly instead via alias. In the init.ora files it had

SPFILE='+DATA/STD11G2/PARAMETERFILE/spfile.257.806251953'               # line added by Agent

and database config also had the

[oracle@rhel6m1 ~]$ srvctl config database -d std11g2
Database unique name: std11g2
Database name: std11g2
Oracle home: /opt/app/oracle/product/11.2.0/dbhome_1
Oracle user: oracle
Spfile: +DATA/STD11G2/PARAMETERFILE/spfile.257.806251953

During the upgrade the DBUA creates a temporary init file in $ORACLE_HOME/dbs and use that to start the upgrade process and later reverting back to spfile. However when it does that following error happens.
Clicking OK would continue but later again fail with following
With this spfile location could be lost and DBUA would be unable to start the database at the end of the upgrade.
To fix this the spfile references were made via an alias.

[oracle@rhel6m1 ~]$ srvctl config database -d std11g2
Database unique name: std11g2
Database name:
Oracle home: /opt/app/oracle/product/11.2.0/dbhome_1
Oracle user: oracle
Spfile: +DATA/STD11G2/spfilestd11g2.ora

How to make an alias for spfile is mentioned in this earlier post.
After these issues were done it was possible to complete upgrade without any further problems.


Once the preupgrade steps are completed, unset ORACLE_HOME and remove from PATH references to old (11.2) oracle home and start the upgrade with DBUA.
Once the progress reaches 100% click on the upgrade result button.
Run the post-upgrade script generated earlier and remove the default listener created earlier (read update on 09/08/2013 below before removing default listener).
Also check the timezone has been upgraded

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
  2      FROM DATABASE_PROPERTIES
  3      WHERE PROPERTY_NAME LIKE 'DST_%'
  4      ORDER BY PROPERTY_NAME;

PROPERTY_NAME         VALUE
-------------------------  --------
DST_PRIMARY_TT_VERSION      18
DST_SECONDARY_TT_VERSION    0
DST_UPGRADE_STATE           NONE

SQL>  SELECT VERSION FROM v$timezone_file;

   VERSION
----------
        18

SQL> select TZ_VERSION from registry$database;

TZ_VERSION
----------
        14

If timezone_file value and value shown in database registry differ then registry could be updated(as per 1509653.1)

SQL> update registry$database set TZ_VERSION = (select version FROM v$timezone_file);

1 row updated.

SQL> commit;

Commit complete.

SQL> select TZ_VERSION from registry$database;

TZ_VERSION
----------
        18

Also the remote listener setting had been reverted back to TCP port instead of the TCPS port setup as part of COST

remote_listener  string   rhel6m-scan:9120

Change it back to use TCPS with SCAN IPs (read update 31/07/2013)

Update 09 August 2013
Check if the listener configured with ASM has changed to default listener.

grid@rhel6m1 ~]$ srvctl config asm
ASM home: /opt/app/12.1.0/grid
Password file:
ASM listener: LISTENER

This would prevent ASM register with the configured non default listener. Change ASM listener configuration before dropping the default listener

[grid@rhel6m1 ~]$ srvctl modify asm -l MYLISTENER

[grid@rhel6m1 ~]$ srvctl config asm
ASM home: /opt/app/12.1.0/grid
Password file:
ASM listener: MYLISTENER

[grid@rhel6m1 ~]$ srvctl stop listener -l listener
[grid@rhel6m1 ~]$ srvctl remove listener -l listener

Useful metalink notes
Oracle Database 12c Release 1 (12.1) DBUA in Silent mode [ID 1516616.1]
Oracle Database 12c Release 1 (12.1) Upgrade New Features [ID 1515747.1]
Complete Checklist for Upgrading to Oracle Database 12c Release 1 using DBUA [ID 1516557.1]
Complete Checklist for Manual Upgrades to Oracle Database 12c Release 1 (12.1) [ID 1503653.1]
Updating the RDBMS DST version in 12c Release 1 (12.1.0.1 and up) using DBMS_DST [ID 1509653.1]
Oracle Database 12c Release 1 (12.1) DBUA : Understanding New Changes With All New 12.1 DBUA [ID 1493645.1]

Related Posts
Upgrading from 11.2.0.3 to 12.1.0.1 RAC
Upgrading from 11gR2 (11.2.0.3) to 12c (12.1.0.1) Grid Infrastructure