Monthly Archives: August 2015

VMWARE VVOL 3PAR and IBM XIV Second Findings

Hello All ,

Before start maybe you need to be sure something about VVOL ;

-> VVOL is object !  http://blogs.vmware.com/education/2015/08/top-questions-vvol-webinar.html

->VMware Virtual Volume API instead of simply presenting a LUN to the hypervisor, letting an ESXi host do data placement and access, a storage system takes on itself a bulk of storage-related functions.

-> Storage Policy Based Management is must

->PE (Protocol Endpoint) what kind a passive multiplexer is it ? Finally understand it from Cormac PE.

  • Protocol Endpoint is a logical I/O proxy presented to a host to communicate with Virtual Volumes stored on a Storage Container. When a virtual machine on the host performs an I/O operation,the protocol endpoint directs the I/O to the appropriate virtual volume. This is a LUN on block storage arrays, and a mount point on NAS arrays. These must be pre-configured on the array, and then presented to all the ESXi hosts that wish to use VVols. These are discovered ‘or’ mounted to the ESXi hosts just like a datastore. However they are not used for storage, just communication.

-> Actually for 3PAR no need to have pre-configured LUN also same on IBM XIV (I mean yourself, they do it for you)

Lets come to IBM VVOL implementation ;

  • You need IBM Spectrum Control Base Edition(SCBE) -> Without it no way to establish control plane between vCenter and IBM XIV (its free)
  • 3PAR VVOL integration completely easy then IBM
    • With IBM you need to install IBM Spectrum Control Base Edition  first
    • Enable Meta Data Service on XIV
    • XIV support Multi-Tenancy (3PAR also have) and before configure VVOL need to create external managed Domain (IBM Spectrum Control Base Edition will manage it) Please google redp5183 , its good and single documentation 😀
    • Then need to care about redundancy of control plane actually ibm have a solution for IBM Spectrum HA Config (You can integrate 3PAR with vCenter as a Storage Provider need only startvasa command on it)
    • Then add the array 😉
      • I have to note here removing storage array from SCBE do not cause any data lost , just only XIV removed from management of IBM Spectrum Control Base Edition
  • Need to set space and service
    • Space just a logical name which can contain service and multiple space provide multiple storage containers
    • Service is about about capability like Thin,Thick and Compression, Encryption and of course about VVOL service
    • Then add new resource/attach resource to service then its over , just only need to add VVOL type datastore

– > About multiple PE ;

What i see , 3PAR support single PE . For IBM XIV because of you are creating ALU for each ESXi node you also provide LUN ID which means you are creating PE for each host. (No documentation but because of we are assigning LUN ID each and ALU is called PE)

IBM created a ALU on the XIV, its is a master LUN, all of the VVOL are luns “behind” the ALU. This is done so we dont exceed the number of paths ESX supports. With the ALU, there is only one set of paths defined, not a set for every VVOL.

-> What is not cool with IBM implementation ;

You should read from notes of SCBE for do not care about meta and tick pools 🙂 but they looks like bad , i don’t know why they do not hide it and giving warning bla bla

Screen Shot 2015-08-20 at 15.41.14

Also you will see very interesting named values there and looks like VVOL and RTM is not working together

Screen Shot 2015-08-20 at 15.46.07

VM

VMWARE VVOL 3PAR and IBM XIV First Findings

Hello All,

Actually a lot of VVOL configuration articles , HP pdfs and IBM XIV Redbook for that but sometimes need more clear explanations.

First you should be sure and double check your HBA firmware, for example i am using Emulex OCe11100 with 10.2.340.18-1OEM.550.0.0.1331820 which is coming with HP VMware 6.0 iso but until upgrade it 10.6.x version i couldn’t see Protocol Endpoint 😦

To learn driver version, there is lovely article from vmware http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1027206

[root@vcd8betahost3:~] esxcfg-scsidevs -a

vmhba0  hpsa              link-n/a  sas.500143802386c9c0                    (0000:03:00.0) Hewlett-Packard Company Smart Array P220i

vmhba1  lpfc              link-up   fc.50060b0000c26e09:50060b0000c26e08    (0000:04:00.2) ServerEngines Corporation Emulex OneConnect OCe11100 FCoE Initiator

vmhba2  lpfc              link-up   fc.50060b0000c26e0b:50060b0000c26e0a    (0000:04:00.3) ServerEngines Corporation Emulex OneConnect OCe11100 FCoE Initiator

vmhba32 usb-storage       link-n/a  usb.vmhba32                             () USB

Important part is “lpfc” for me , then

[root@vcd8betahost3:~] vmkload_mod -s lpfc | grep Version

Version: 10.6.126.0-1OEM.600.0.0.2159203

For Emulex http://www.emulex.com/downloads/emulex/drivers/vmware/vsphere-60/drivers/ you find out driver from link and just standard vib or offline-bundle installation is required lie below ;

esxcli software vib install -v {VIBFILE}  (here replace {VIBFILE} with lpfc-10.6.126.0-1OEM.600.0.0.2159203.x86_64.vib)

or

esxcli software vib install -d {OFFLINE_BUNDLE} (here replace {OFFLINE_BUNDLE} with VMW-ESX-6.0.0-lpfc-10.6.126.0-offline_bundle-2821510.zip)

its take a while for me , if something wrong pls check “esxupdate.log” from ESX node

PE(Protocol Endpoint) known as a Proxy , IBM call ALU (Administrative Logical Unit)

Also check firmware requirements from this pdf for HP page 7 http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA5-6907ENW.pdf and for IBM XIV 11.6.0.b

i guess 3PAR VVOL configuration easier and more understandable then IBM XIV For example no need to create additional domain , pool and create manual administrative logical unit which 3PAR create automatically, just only need be sure VASA service is running on 3PAR.

Be sure Protocol Endpoint is running , if service is disabled then execute “startvasa” command

HOSTNAME3PAR cli% showvasa

-Service- ———VASA_API2_URL——— -MemUsage(MB)- -Version-

Enabled   https://10.111.14.100:9997/vasa            339 2.1.6

For IBM XIV (i guess it will be also needed for Storewize) you need to extra install “IBM Spectrum Control Base” software, its kind a thin version and of course free version with some functionality of IBM Tivoli. Its looks like developed with python django framework. After register XIV and then create VVOL Service and Resource on it and register VASA Storage Provider on vCenter then its okay to go.

Most important thing is checking the log , its all “/var/log/sc” folder and from events.log you can find out many things. (Pls check all others for learn and see different things)

To install and troubleshot pls download IBM_Spectrum_Control_Base_2.1.0_UG

Example view of it

Screen Shot 2015-08-11 at 00.53.51

You will mostly see that every article will inform you to execute “esxcli core storage device list –pe-only” but here pls do not forget for example if VASA service on 3PAR is not running or something wrong with HBA,Storage firmware then you can not see something and if everything is success additional to your storage device which vmware aware about it you will have another extra new device with same name but “Is VVOL PE ” is  true , like below

   Display Name: 3PARdata Fibre Channel Disk (naa.2ff70002ac00cb53)

Vendor: 3PARdata

Is VVOL PE: true

Display Name: 3PARdata Fibre Channel Disk (naa.60002ac0000000000000000a0000cb53)

Vendor: 3PARdata
Is VVOL PE: false

For 3PAR you will see some capacity assigned to VVOL PE enables device which is 512 B don’t worry

Like VSAN you have to create VM Storage Profile , here some documentations show different thing but for VASA 2.0 we should use com.hp.3par.spbm for data service.

About the rule sets  you will see some screenshots like below

Screen Shot 2015-08-11 at 01.16.43

 

but for me its not like that at the startup more strong and long lines but after logout and login vCenter Web Clients its fixed like above 😀

Some notes for me , to do not forgot something about 3PAR command line

Also i made some snapshot test and i can see that still Checkpoint_Unstun there 😦

3PAR Host Creation

First need to define host but before need to learn host types (Actually on XIV generally we set Generic if its not other IBM device);

HOSTNAME3PAR cli% showhost -listpersona

Persona_Id Persona_Name   Persona_Caps

1 Generic        UARepLun,SESLun

2 Generic-ALUA   UARepLun,SESLun,ALUA

6 Generic-legacy —

7 HPUX-legacy    VolSetAddr,Lun0SCC

8 AIX-legacy     NACA

9 EGENERA        SoftInq

10 ONTAP-legacy   SoftInq

11 VMware         SubLun,ALUA

12 OpenVMS        UARepLun,RTPG,SESLun,Lun0SCC

13 HPUX           UARepLun,VolSetAddr,SESLun,ALUA,Lun0SCC

15 WindowsServer  UARepLun,SESLun,ALUA,WSC

Then create host and set persona but before check WWPN appeared by 3PAR ;

HOSTNAME3PAR cli% showhost

Id Name Persona -WWN/iSCSI_Name- Port

0 ESX  VMware  50060B0000C26E08 1:1:1

—              50060B0000C26220 1:1:1

500143802668E844 1:1:1

Creating host ;

createhost ESX 50060B0000C26E08

Related command set persona value default 2 pls set it 11 and what i saw when active i can not set it to 11 which is support ALUA and reset the node when its offline set it for 11 (pls test with your 3par maybe no needed to offline node)

“sethost -persona 11 -os VMware ESX” (ESX is hostname also this command from HP docs , i push this command without -os paramter)

To see the created/assigned virtual volumes to me (don’t mix it with 3PAR virtual volumes) execute fallowing command below ; Via API communications storage also aware about VMs 😀

HOSTNAME3PAR cli% showvvolvm

——(MB)——

VM_Name Num_vv Physical Logical GuestOS               State

VVOLVM1      3    30771   49152 windows8Server64Guest Bound

HOSTNAME3PAR cli% showvvolvm -vv

——(MB)——

VM_Name VV_ID vVol_Name             vVol_Type Prov Physical Logical

VVOLVM1    12 cfg-VVOLVM1-4cbe8a29  Config    tpvv     3200    4096

13 dat-VVOLVM1.-be812edd Data      tpvv    18560   40960

14 swp-VVOLVM1f-fcf1f816 Swap      full     4915    4096

For Stats ;

HOSTNAME3PAR cli% statvv dat-VVOLVM1.-be812edd

13:15:50 08/17/2015 r/w I/O per second KBytes per sec    Svt ms IOSz KB

VVname      Cur  Avg  Max  Cur  Avg  Max  Cur  Avg Cur Avg Qlen

dat-VVOLVM1.-be812edd   t    0    0    0    0    0    0 0.00 0.00 0.0 0.0    0

——————————————————————————

1   t    0    0         0    0      0.00 0.00 0.0 0.0    0

3PAR CLI Commands http://etactiks.blogspot.com.tr/2013/02/hp-3par-cli-command-list.html

Other good article from HP for implementation http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA5-6907ENW.pdf