最近一个客户一套11.2.0.4环境的rac一周内掉盘掉了2次找人查看硬件也说没有报错刚好有时间抽空下来好好的看看日志1、出现掉盘监控告警asm空间异常登录系统看果然有问题显示一块磁盘offline。2、查看asm日志这个asm日志很大而且一直在刷重复的信息感觉是踩了bug。NOTE: Voting file relocation is required in diskgroup OCRVOTENOTE: Attempting voting file relocation on diskgroup OCRVOTENOTE: Successful voting file relocation on diskgroup OCRVOTE翻了好久才有点端倪凌晨4点的时候告警WARNING: Waited 15 secs for write IO to PST disk 1 in group 3.WARNING: Waited 15 secs for write IO to PST disk 1 in group 3.Wed Nov 27 04:40:39 2024NOTE: process _b000_asm2 (49704) initiating offline of disk 1.3915955794 (DATA4) with mask 0x7e in group 3NOTE: checking PST: grp 3GMON checking disk modes for group 3 at 44 for pid 39, osid 49704NOTE: group DATA2: updated PST location: disk 0000 (PST copy 0)NOTE: checking PST for grp 3 done.NOTE: sending set offline flag message 1401034769 to 1 disk(s) in group 3WARNING: Disk DATA4 in mode 0x7f is now being offlinedNOTE: initiating PST update: grp 3, dsk 1/0xe968be52, mask 0x6a, op clearGMON updating disk modes for group 3 at 45 for pid 39, osid 49704NOTE: group DATA2: updated PST location: disk 0000 (PST copy 0)NOTE: group DATA2: updated PST location: disk 0000 (PST copy 0)一看15s这个告警结合11g的环境想到了_asm_hbeatiowait这个参数这个参数是11.2.0.3推出来的用于心跳磁盘超时检测超时15s未响应就会踢出磁盘或者dismount磁盘组很庆幸这次只是drop一个磁盘业务没有受到影响。这个参数在12c默认值就很大了达到120s在19c就更大。可以通过以下语句修改sqlplus / as sysasmalter system set _asm_hbeatiowait120s scopespfile sid*;3、加回磁盘--查看磁盘状态select GROUP_NUMBER,NAME,PATH,MOUNT_STATUS,STATE from v$asm_disk;--确定好掉的盘后加盘kfod disksall statusTRUE dscvgroupTRUEkfed read /dev/磁盘 |grep kfdhdb.dsknamealter diskgroup data add disk ORCL:DATA4 force;--查看重平衡select operation,est_minutes from v$asm_operation当est_minutes为0时表示完成。结束最后再找个时间修改下隐藏参数。