ORA-15477 cannot communicate with the volume driver

ORA-15477: cannot communicate with the volume driver could be seen when setting the compatible.admv value on a ASM disk group. The system was running 11.2.0.3 (11.2.0.3.4) RAC on RHEL 6 (2.6.32-220.el6.x86_64) and it's a brand new installation of RAC (1488004.1).

SQL> alter diskgroup flash set attribute 'compatible.advm'='11.2';
alter diskgroup flash set attribute 'compatible.advm'='11.2'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15242: could not set attribute compatible.advm
ORA-15238: 11.2 is not a valid value for attribute compatible.advm
ORA-15477: cannot communicate with the volume driver

It seems the ACFS/ADMV modules are not loaded with system reboot as

[root@rhel6m1 ~]# lsmod | grep oracle

didn't return anything.
Reinstall acfs/admv modules manually (1371067.1)

[root@rhel6m1 bin]# ./acfsroot install
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9312: Existing ADVM/ACFS installation detected.
ACFS-9314: Removing previous ADVM/ACFS installation.
ACFS-9315: Previous ADVM/ACFS components successfully removed.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9309: ADVM/ACFS installation correctness verified.

After this acfs modules will be loaded

lsmod  | grep oracle
oracleacfs           1857627  2
oracleadvm            238976  6
oracleoks             330757  2 oracleacfs,oracleadvm

and change of compatible.admv would proceed without an error and acfs volumes could be created

SQL> alter diskgroup flash set attribute 'compatible.advm'='11.2';
Diskgroup altered.

SQL> alter diskgroup flash add volume volume1 size 1g;
Diskgroup altered.

SQL> select volume_name,volume_device from v$asm_volume;

VOLUME_NAME                    VOLUME_DEVICE
------------------------------ --------------------
VOLUME1                        /dev/asm/volume1-149

# mkfs -t acfs /dev/asm/volume1-149
mkfs.acfs: version                   = 11.2.0.3.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/volume1-149
mkfs.acfs: volume size               = 1073741824
mkfs.acfs: Format complete.

acfsutil registry -a /dev/asm/volume1-149 /opt/acfsvol

#mount -t acfs  /dev/asm/volume1-149 /opt/acfsvol

However after the reboot of the servers the acfs modules do not get auto loaded and following could be seen on ASM alert log

SUCCESS: ALTER DISKGROUP ALL MOUNT /* asm agent call crs *//* {0:0:2} */
SQL> ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:0:2} */
kfvxVolOnOff: Cannot open device file
ORA-15032: not all alterations performed
ORA-15477: cannot communicate with the volume driver
ERROR: ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:0:2} */

To fix this run acfsroot enable. This command requires acfs to be started before running it

[root@rhel6m1 grid]# $GI_HOME/bin/acfsroot enable
ACFS-9167: ADVM/ACFS is not installed or loaded. Run 'acfsroot install'.

[root@rhel6m1 grid]# $GI_HOME/bin/acfsload start -s
[root@rhel6m1 grid]# $GI_HOME/bin/acfsroot enable
ACFS-9376: Adding ADVM/ACFS drivers resource succeeded.
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rhel6m1'
CRS-2676: Start of 'ora.drivers.acfs' on 'rhel6m1' succeeded
ACFS-9380: Starting ADVM/ACFS drivers resource succeeded.
ACFS-9368: Adding ACFS registry resource succeeded.
CRS-2672: Attempting to start 'ora.registry.acfs' on 'rhel6m1'
CRS-2676: Start of 'ora.registry.acfs' on 'rhel6m1' succeeded
ACFS-9372: Starting ACFS registry resource succeeded.

This issue is mentioned in (1488004.1 and 1486208.1)
Afterwards acfs modules were auto loading on reboot and acfs volumes were getting auto mounted without a problem.

Useful metalink notes
ACFS Supported On OS Platforms. [ID 1369107.1]
ACFS Filesystem Not Mounted After Host Reboot [ID 1473875.1]
How To Install/Reinstall Or Deinstall ACFS Modules/Installation Manually? [ID 1371067.1]
11.2.0.3.3 Grid Infrastructure Patchset: 13919095 Installation On RH 6.3 Fails With ACFS-9459 . [ID 1493491.1]
ORA.REGISTRY.ACFS Resource Appears As OFFLINE On A Brand New 11.2 RAC Installation/Configuration. [ID 1488004.1]
ACFS/ADVM is NOT started automatically after node reboot or after CRS is restarted in non-RAC environment [ID 886407.1]
“ora.drivers.acfs” Resource Was Not Configured Therefore RAC ACFS Filesystem Is Not Mounting During The Reboot. [ID 1486208.1]

Related Post
Creating and Removing ACFS