{"id":422,"date":"2013-07-09T16:58:15","date_gmt":"2013-07-09T19:58:15","guid":{"rendered":"http:\/\/www.dbarj.com.br\/?p=422"},"modified":"2014-10-09T23:38:38","modified_gmt":"2014-10-10T02:38:38","slug":"oracle-rac-vmware-ora-27072","status":"publish","type":"post","link":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/","title":{"rendered":"Oracle RAC in VMWare hangs with ORA-27072: File I\/O error"},"content":{"rendered":"<p>During three months, I have had constant problems of an Oracle RAC installation suddenly hanging. The system was as follows:<\/p>\n<ul>\n<li>O.S: Red Hat Enterprise Linux Server release 5.8 (Tikanga) &#8211; x86_64<\/li>\n<li>Kernel: 2.6.18-308.16.1.el5<\/li>\n<li>VMWare: VMware ESXi 5.0<\/li>\n<li>Oracle 11.2.0.3 PSU 6<\/li>\n<li>Oracle homes in different drives and shared disks for RAC via VMFS<\/li>\n<\/ul>\n<p>It was random, the node dropped and came back online. Sometimes the machine restarted by Oracle processes.<br \/>\nEven after re-installing, upgrading Oracle to the latest version and applying the latest PSU, the problem continued.<\/p>\n<p>The oracle alert and trace files showed the following error:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">ORA-27072: File I\/O error\r\nLinux-x86_64 Error: 5: Input\/output error\r\nAdditional information: 4\r\nAdditional information: 657408\r\nAdditional information: -1\r\nWARNING: Read Failed. group:1 disk:0 AU:321 offset:0 size:4096\r\npath:\/dev\/oracleasm\/disks\/DATA01\r\nincarnation:0xe968aff8 synchronous result:'I\/O error'\r\nsubsys:System iop:0x7fb195be9000 bufp:0x7fb196117000 osderr:0x0 osderr1:0x0\r\nERROR: failed to read ACD block gn=1 blk=10752\r\nORA-15080: synchronous I\/O operation to a disk failed\r\nWARNING: LGWR failed to read ACDC for diskgroup 1 thread 2\r\nWARNING: disk offlining resulting in I\/O error\r\nWARNING: Write Failed. group:1 disk:0 AU:321 offset:0 size:4096\r\npath:\/dev\/oracleasm\/disks\/DATA01\r\nincarnation:0xe968aff8 asynchronous result:'I\/O error'\r\nsubsys:System iop:0x7fb195be9000 bufp:0x647fd000 osderr:0x534b4950 osderr1:0x0<\/pre>\n<p>Searching further, I found the following error in the log in Redhat \/var\/log\/messages:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">Jul 6 19:45:58 oraclesrv01 kernel: sd 1:0:0:0: reservation conflict\r\nJul 6 19:45:58 oraclesrv01 kernel: sd 1:0:0:0: Unhandled error code\r\nJul 6 19:45:58 oraclesrv01 kernel: sd 1:0:0:0: SCSI error: return code = 0x00110018\r\nJul 6 19:45:58 oraclesrv01 kernel: Result: hostbyte=invalid driverbyte=DRIVER_OK,SUGGEST_OK<\/pre>\n<p>As a workaround, I tried:<\/p>\n<ul>\n<li>Mount brand new discs, but the problem remains.<\/li>\n<li>Separate the data in one disk and the cluster configuration and votefile in another 3 disks.<\/li>\n<\/ul>\n<p>Nothing solved. After a lot of research, I finally discovered the cause of the error.<\/p>\n<p><span style=\"color: #0000ff;\">VMFS is a clustered file system that disables (by default) multiple virtual machines from opening and writing to the same virtual disk (.vmdk file). This prevents more than one virtual machine from inadvertently accessing the same .vmdk file.<\/span><\/p>\n<p><span style=\"color: #0000ff;\">The multi-writer option allows VMFS-backed disks to be shared by multiple virtual machines. This option is used to support VMware fault tolerance, which allows a primary virtual machine and a standby virtual machine to simultaneously access a .vmdk file.<\/span><\/p>\n<p>The article link below teaches you step by step to activate this option:<\/p>\n<p><a title=\"http:\/\/kb.vmware.com\/selfservice\/microsites\/search.do?language=en_US&amp;cmd=displayKC&amp;externalId=1034165\" href=\"http:\/\/kb.vmware.com\/selfservice\/microsites\/search.do?language=en_US&amp;cmd=displayKC&amp;externalId=1034165\">http:\/\/kb.vmware.com\/selfservice\/microsites\/search.do?language=en_US&amp;cmd=displayKC&amp;externalId=1034165<\/a><br \/>\n<strong>Disabling simultaneous write protection provided by VMFS using the multi-writer flag (1034165)<\/strong><\/p>\n<p>Briefly, what needs to be done is to edit the file &#8220;.vmx&#8221; that defines the settings of the virtual machine and add for each shared disk the following entry:<\/p>\n<p style=\"padding-left: 30px;\">scsiX:Y.sharing = &#8220;multi-writer&#8221;<\/p>\n<p>If four disks are being shared, the entries would be as follows:<\/p>\n<p style=\"padding-left: 30px;\">scsi1:0.sharing = &#8220;multi-writer&#8221;<br \/>\nscsi1:1.sharing = &#8220;multi-writer&#8221;<br \/>\nscsi1:2.sharing = &#8220;multi-writer&#8221;<br \/>\nscsi1:3.sharing = &#8220;multi-writer&#8221;<\/p>\n<p>That&#8217;s it! Now your oracle will stop giving this inexplicable error.<\/p>\n<b>Have you enjoyed? Please leave a comment or give a \ud83d\udc4d!<\/b>\n<div class='watch-action'><div class='watch-position align-left'><div class='action-like'><a class='lbg-style2 like-422 jlk' href='javascript:void(0)' data-task='like' data-post_id='422' data-nonce='de4404f630' rel='nofollow'><img class='wti-pixel' src='https:\/\/www.dbarj.com.br\/wp-content\/plugins\/wti-like-post\/images\/pixel.gif' title='Like' \/><span class='lc-422 lc'>+10<\/span><\/a><\/div><\/div> <div class='status-422 status align-left'><\/div><\/div><div class='wti-clear'><\/div>","protected":false},"excerpt":{"rendered":"<p>During three months, I have had constant problems of an Oracle RAC installation suddenly hanging. The system was as follows: O.S: Red Hat Enterprise Linux Server release 5.8 (Tikanga) &#8211; x86_64 Kernel: 2.6.18-308.16.1.el5 VMWare: VMware ESXi 5.0 Oracle 11.2.0.3 PSU 6 Oracle homes in different drives and shared disks for RAC via VMFS It was &hellip; <\/p>\n<p><a class=\"more-link btn\" href=\"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/\">Continue reading<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25],"tags":[],"class_list":["post-422","post","type-post","status-publish","format-standard","hentry","category-linux-en","item-wrap"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Oracle RAC in VMWare hangs with ORA-27072: File I\/O error - DBA - Rodrigo Jorge - Oracle Tips and Guides<\/title>\n<meta name=\"description\" content=\"Resolved: why Oracle RAC in VMWare hangs with ORA-27072.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DBA RJ\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/\"},\"author\":{\"name\":\"DBA RJ\",\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/#\\\/schema\\\/person\\\/28a44ca3a6633fe4156ad1ea209d40a9\"},\"headline\":\"Oracle RAC in VMWare hangs with ORA-27072: File I\\\/O error\",\"datePublished\":\"2013-07-09T19:58:15+00:00\",\"dateModified\":\"2014-10-10T02:38:38+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/\"},\"wordCount\":354,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/#\\\/schema\\\/person\\\/28a44ca3a6633fe4156ad1ea209d40a9\"},\"articleSection\":[\"Linux for Database\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/\",\"url\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/\",\"name\":\"Oracle RAC in VMWare hangs with ORA-27072: File I\\\/O error - DBA - Rodrigo Jorge - Oracle Tips and Guides\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/#website\"},\"datePublished\":\"2013-07-09T19:58:15+00:00\",\"dateModified\":\"2014-10-10T02:38:38+00:00\",\"description\":\"Resolved: why Oracle RAC in VMWare hangs with ORA-27072.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/2013\\\/07\\\/oracle-rac-vmware-ora-27072\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Oracle RAC in VMWare hangs with ORA-27072: File I\\\/O error\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/\",\"name\":\"DBA - Rodrigo Jorge - Oracle Tips and Guides\",\"description\":\"Blog about Databases, Security and High Availability\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/#\\\/schema\\\/person\\\/28a44ca3a6633fe4156ad1ea209d40a9\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/en\\\/#\\\/schema\\\/person\\\/28a44ca3a6633fe4156ad1ea209d40a9\",\"name\":\"DBA RJ\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/wp-content\\\/uploads\\\/2019\\\/09\\\/RodrigoJorgePOUG19.png\",\"url\":\"https:\\\/\\\/www.dbarj.com.br\\\/wp-content\\\/uploads\\\/2019\\\/09\\\/RodrigoJorgePOUG19.png\",\"contentUrl\":\"https:\\\/\\\/www.dbarj.com.br\\\/wp-content\\\/uploads\\\/2019\\\/09\\\/RodrigoJorgePOUG19.png\",\"width\":712,\"height\":712,\"caption\":\"DBA RJ\"},\"logo\":{\"@id\":\"https:\\\/\\\/www.dbarj.com.br\\\/wp-content\\\/uploads\\\/2019\\\/09\\\/RodrigoJorgePOUG19.png\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Oracle RAC in VMWare hangs with ORA-27072: File I\/O error - DBA - Rodrigo Jorge - Oracle Tips and Guides","description":"Resolved: why Oracle RAC in VMWare hangs with ORA-27072.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/","twitter_misc":{"Written by":"DBA RJ","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/#article","isPartOf":{"@id":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/"},"author":{"name":"DBA RJ","@id":"https:\/\/www.dbarj.com.br\/en\/#\/schema\/person\/28a44ca3a6633fe4156ad1ea209d40a9"},"headline":"Oracle RAC in VMWare hangs with ORA-27072: File I\/O error","datePublished":"2013-07-09T19:58:15+00:00","dateModified":"2014-10-10T02:38:38+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/"},"wordCount":354,"commentCount":0,"publisher":{"@id":"https:\/\/www.dbarj.com.br\/en\/#\/schema\/person\/28a44ca3a6633fe4156ad1ea209d40a9"},"articleSection":["Linux for Database"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/","url":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/","name":"Oracle RAC in VMWare hangs with ORA-27072: File I\/O error - DBA - Rodrigo Jorge - Oracle Tips and Guides","isPartOf":{"@id":"https:\/\/www.dbarj.com.br\/en\/#website"},"datePublished":"2013-07-09T19:58:15+00:00","dateModified":"2014-10-10T02:38:38+00:00","description":"Resolved: why Oracle RAC in VMWare hangs with ORA-27072.","breadcrumb":{"@id":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbarj.com.br\/en\/2013\/07\/oracle-rac-vmware-ora-27072\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.dbarj.com.br\/en\/"},{"@type":"ListItem","position":2,"name":"Oracle RAC in VMWare hangs with ORA-27072: File I\/O error"}]},{"@type":"WebSite","@id":"https:\/\/www.dbarj.com.br\/en\/#website","url":"https:\/\/www.dbarj.com.br\/en\/","name":"DBA - Rodrigo Jorge - Oracle Tips and Guides","description":"Blog about Databases, Security and High Availability","publisher":{"@id":"https:\/\/www.dbarj.com.br\/en\/#\/schema\/person\/28a44ca3a6633fe4156ad1ea209d40a9"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbarj.com.br\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.dbarj.com.br\/en\/#\/schema\/person\/28a44ca3a6633fe4156ad1ea209d40a9","name":"DBA RJ","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbarj.com.br\/wp-content\/uploads\/2019\/09\/RodrigoJorgePOUG19.png","url":"https:\/\/www.dbarj.com.br\/wp-content\/uploads\/2019\/09\/RodrigoJorgePOUG19.png","contentUrl":"https:\/\/www.dbarj.com.br\/wp-content\/uploads\/2019\/09\/RodrigoJorgePOUG19.png","width":712,"height":712,"caption":"DBA RJ"},"logo":{"@id":"https:\/\/www.dbarj.com.br\/wp-content\/uploads\/2019\/09\/RodrigoJorgePOUG19.png"}}]}},"_links":{"self":[{"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/posts\/422","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/comments?post=422"}],"version-history":[{"count":0,"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/posts\/422\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/media?parent=422"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/categories?post=422"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbarj.com.br\/en\/wp-json\/wp\/v2\/tags?post=422"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}