jun 18

Migrando OCR e Voting Disk para Oracle ASM Online

This post is also available in: English

Antes da versão Oracle 11gR2, só era possível termos os arquivos de OCR e Voting Disk em múltiplos raw devices, compartilhados entre os nós de um Cluster.
A partir desta versão, se torna bem interessante aproveitarmos dos recursos de mirror e striping do ASM para colocarmos estes arquivos.

Para colocarmos o OCR e Voting Disk em um diskgroup, devemos obedecer as seguintes regras no momento de definição:

  • Se o seu diskgroup tive redundância "external", será necessário que ele tenha no mínimo 1 disco associado.
  • Se o seu diskgroup tive redundância "normal", será necessário que ele tenha no mínimo 3 discos associados, mesmo sendo possível criar esse diskgroup com apenas 2 discos.
  • Se o seu diskgroup tive redundância "high", será necessário que ele tenha no mínimo 5 discos associados, mesmo sendo possível criar esse diskgroup com apenas 3 discos.

Neste artigo, irei utilizar a segunda opção, de redundância normal. Como no meu servidor só existem 2 discos disponíveis para a criação, precisei solicitar a equipe de Infra a adição de mais um disco de 1 GB:

 
[root@orarac01 ~]# /etc/init.d/oracleasm listdisks
DATA01
OCRVOTE01
OCRVOTE02
[root@orarac01 ~]#

O disco já foi disponibilizado:

 
[root@orarac01 ~]# multipath -ll
DATA01 (36000c29ab0accd959da8f1f78dd2de2f) dm-4 VMware,Virtual disk
size=300G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 3:0:2:0 sde 8:64 active ready running
OCRVOTE03 (36000c299550e110b041c9ab26acd33ac) dm-5 VMware,Virtual disk
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 3:0:3:0 sdf 8:80 active ready running
OCRVOTE02 (36000c29b67329567f6a6547d6fd3c769) dm-2 VMware,Virtual disk
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 3:0:0:0 sdc 8:32 active ready running
OCRVOTE01 (36000c29c1a57ee047d443991ef48894a) dm-3 VMware,Virtual disk
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 3:0:1:0 sdd 8:48 active ready running
[root@orarac01 ~]#

Adicionando um disco novo para o oracleasm:

 
[root@orarac01 ~]# /etc/init.d/oracleasm createdisk OCRVOTE03 /dev/dm-5
Marking disk "OCRVOTE03" as an ASM disk:                   [  OK  ]
[root@orarac01 ~]# /etc/init.d/oracleasm listdisks
DATA01
OCRVOTE01
OCRVOTE02
OCRVOTE03
[root@orarac01 ~]#

Habilitando o disco no outro nó do RAC:

 
[root@orarac02 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@orarac02 ~]# /etc/init.d/oracleasm listdisks
DATA01
OCRVOTE01
OCRVOTE02
OCRVOTE03
[root@orarac02 ~]#

O próximo passo é criarmos o novo diskgroup no ASM.

Primeiro, vamos tentar colocá-los como "High Redundancy" e ver o que acontece.

 
CREATE DISKGROUP OCRVOTE HIGH REDUNDANCY
FAILGROUP OCRVOTE_FG01 DISK '/dev/oracleasm/disks/OCRVOTE01' NAME OCRVOTE01  SIZE 1024M
FAILGROUP OCRVOTE_FG02 DISK '/dev/oracleasm/disks/OCRVOTE02' NAME OCRVOTE02  SIZE 1024M
FAILGROUP OCRVOTE_FG03 DISK '/dev/oracleasm/disks/OCRVOTE03' NAME OCRVOTE03  SIZE 1024M
ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M';
 
[root@orarac01 ~]# crsctl replace votedisk +OCRVOTE
Failed to create voting files on disk group OCRVOTE.
Change to configuration failed, but was successfully rolled back.
CRS-4000: Command Replace failed, or completed with errors.
[root@orarac01 ~]#
 
[root@orarac01 ~]# tail -n 200 $GRID_HOME/log/orarac01/client/crsctl_root.log
...
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: === clsssConfigLock ===
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: lock(0x2838a50), version(1), size(1088)
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: offsets(24), activever(186647296)
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: id(15), instantiation(5), incarn(1)
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: mapoff(28), configoff(548), mapsize(512)
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: state(10), holders(1), waiters(0)
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: committimestamp(0), commitstate(0)
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: === Map (of first 7 entries) ===
2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: 010 000 000 000 000 000 000
2013-06-18 11:46:13.221: [  CRSCTL][278546160]crsctl_replace_votedisk: calling kgfdvfPushDiskString
2013-06-18 11:46:13.397: [  CRSCTL][278546160]function crsctl_replace_votedisk, error 1 [clsuSlosFormatDiag called with non-error slos.]
.

2013-06-18 11:46:13.397: [  CRSCTL][278546160]crsctl_replace_votedisk: pushed the ASM DS to the profile 1
[   CLWAL][278546160]clsw_Initialize: OLR initlevel [30000]
2013-06-18 11:46:13.432: [  CRSCTL][278546160]crsctl_format_diskgroup: creating diskgroup OCRVOTE
2013-06-18 11:46:16.555: [  CRSCTL][278546160]function crsctl_format_diskgroup, error 1 SLOS: cat=-1, opn=(none), dep=(none), loc=ORA-01405: f
.

2013-06-18 11:46:16.555: [  CRSCTL][278546160]crsctl_format_diskgroup: diskgroup OCRVOTE creation with status 1. Please check the alert log file for ASM
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: === clsssConfigUnlock ===
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: lock(0x2837b50), version(1), size(1088)
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: offsets(24), activever(186647296)
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: id(15), instantiation(5), incarn(3)
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: mapoff(28), configoff(548), mapsize(512)
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: state(0), holders(0), waiters(0)
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: committimestamp(0), commitstate(0)
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: === Map (of first 7 entries) ===
2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: 000 000 000 000 000 000 000
...
 
[root@orarac01 ~]# tail -n 500 /u01/app/grid/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log
...
Tue Jun 18 11:46:13 2013
NOTE: Creating voting files in diskgroup OCRVOTE
Tue Jun 18 11:46:13 2013
NOTE: Voting File refresh pending for group 2/0x7f386577 (OCRVOTE)
NOTE: Attempting voting file creation in diskgroup OCRVOTE
NOTE: voting file allocation on grp 2 disk OCRVOTE_0000
NOTE: voting file allocation on grp 2 disk OCRVOTE_0001
NOTE: voting file allocation on grp 2 disk OCRVOTE_0002
ERROR: Voting file allocation failed for group OCRVOTE
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_6301.trc:
ORA-15274: Not enough failgroups (5) to create voting files
NOTE: Attempting voting file refresh on diskgroup OCRVOTE
NOTE: Voting file relocation is required in diskgroup OCRVOTE
NOTE: Attempting voting file relocation on diskgroup OCRVOTE
NOTE: voting file deletion on grp 2 disk OCRVOTE_0000
NOTE: voting file deletion on grp 2 disk OCRVOTE_0001
NOTE: voting file deletion on grp 2 disk OCRVOTE_0002
...

O erro é que são necessários pelo menos 5 failgroups para criar o Voting Disk.

Vamos recriá-los então com a redundância correta:

 
CREATE DISKGROUP OCRVOTE NORMAL REDUNDANCY
FAILGROUP OCRVOTE_FG01 DISK '/dev/oracleasm/disks/OCRVOTE01' NAME OCRVOTE01  SIZE 1024M
FAILGROUP OCRVOTE_FG02 DISK '/dev/oracleasm/disks/OCRVOTE02' NAME OCRVOTE02  SIZE 1024M
FAILGROUP OCRVOTE_FG03 DISK '/dev/oracleasm/disks/OCRVOTE03' NAME OCRVOTE03  SIZE 1024M
ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M';
 
[root@orarac01 ~]# crsctl replace votedisk +OCRVOTE
Successful addition of voting disk 05ca1320e3244f70bf45d1fc37d325d5.
Successful addition of voting disk 1affc0b6da314fe8bfd8dec18af4923b.
Successful addition of voting disk 3d87bf2793e04fe4bfd6764f24ffafb1.
Successful deletion of voting disk 4c9ddcb1acd14f27bf77652370a1a5b1.
Successfully replaced voting disk group with +OCRVOTE.
CRS-4266: Voting file(s) successfully replaced
[root@orarac01 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   05ca1320e3244f70bf45d1fc37d325d5 (/dev/oracleasm/disks/OCRVOTE01) [OCRVOTE]
 2. ONLINE   1affc0b6da314fe8bfd8dec18af4923b (/dev/oracleasm/disks/OCRVOTE02) [OCRVOTE]
 3. ONLINE   3d87bf2793e04fe4bfd6764f24ffafb1 (/dev/oracleasm/disks/OCRVOTE03) [OCRVOTE]
Located 3 voting disk(s).
[root@orarac01 ~]#

O Voting Disk já foi.
O próximo passo é o CRS. Note que estamos transferindo ele de outro diskgroup (+DATA), que é utilizado para guardar datafiles, para este recém criado:

 
[root@orarac01 ~]# ocrconfig -replace +DATA -replacement +OCRVOTE
PROT-28: Cannot delete or replace the only configured Oracle Cluster Registry location
[root@orarac01 ~]#

Por existir apenas um destino OCR configurado, o comando "replace" não funciona. Vamos precisar adicionar um outro e apagar o antigo:

 
[root@orarac01 ~]# ocrconfig -add +OCRVOTE
[root@orarac01 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2836
         Available space (kbytes) :     259284
         ID                       : 1089167638
         Device/File Name         :      +DATA
                                    Device/File integrity check succeeded
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

[root@orarac01 ~]#

Agora apagamos o antigo:

 		 
[root@orarac01 ~]# ocrconfig -delete +DATA
[root@orarac01 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2836
         Available space (kbytes) :     259284
         ID                       : 1089167638
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

[root@orarac01 ~]# cat /etc/oracle/ocr.loc
#Device/file +DATA getting replaced by device +OCRVOTE
ocrconfig_loc=+OCRVOTE
local_only=false
[root@orarac01 ~]#

Pronto!

Mais informações podem ser obtidas no Artigo do Metalink abaixo:
OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE) [ID 428681.1]

Gostou? Não deixe de comentar ou deixar um 👍!

 

Deixe um comentário

Seu e-mail não será publicado.