Since vSphere 5.0 I have gotten a lot of question whether a customer should upgrade current VMFS3 to VMFS5 or delete them and re-create them from scratch. My normal response to this question has been: If you have the time storage space to empty one datastore at a time and re-create it then do it. Then you would get the unified block size of 1MB on all datastores and operations like deploying from templates and storage vmotion would work more efficiently. But if you don’t have the time don’t worry about it.
But after reading VMWare’s newly released VAAI white paper is has come to my attention that upgraded VMFS3 volumes does not use all 8 VAAI ATS (Atomic Test and Set) Primitives to prevent scsi reservation. Upgraded VMFS3 only use Primitive 1 and 2. The 8 primitives are:
1. Acquire on-disk locks
2. Upgrade an optimistic lock to an exclusive/physical lock
3. Unlock a read-only/multiwriter lock
4. Acquire a heartbeat
5. Clear a heartbeat
6. Replay a heartbeat
7. Reclaim a heartbeat
8. Acquire on-disk lock with dead owner
ATS or Atomic Test and Set is a very important feature in VMware to prevent locking features. Especially today where you see most customers increasing their datastores to several Terabytes. By increasing the size you also increase the number of Virtual Machines on each datastore and increase the chance of problem with locking.
So my recommendation today is: If you use VAAI and your storage array supports is and you are running vsphere Enterprise og Enterprise Plus licenses you should recreate all datastores from scratch. This will set the flag “ATS only” as True and you will have all locking primitives enabled.
The only LUNS that are showing latency issues are LUNS created from scratch as VMFS v5. They have ATS Only set as mode and have huge storage latency issues. The LUNS I upgraded from version 3 to 5, and have mode set to public have no latency issues. I can’t even turn VAAI off to test because of this “Mode: Public ATS-Only” is set on the new VMFS 5 volumes. The ones with ATS-Only weren’t even upgraded from version 3 of VMFS, they were created from scratch as VMFS 5. I have an AMS 2100 and Hitachi has recommended turning VAAI off as this model SAN even though it supports it wasn’t built to handle it when VAAI came out.