PowerCLI: Balance LUN paths for a Cluster

As I’ve been managing more and more infrastructures using fibre channel storage, I’ve found that it’s been somewhat difficult to keep the LUN paths to each host balanced. By balanced, I mean that for each LUN to each host, there is a number of paths and I want to make sure that, for example, each LUN 10 to each of the hosts is using path A as the primary and path B as the stand by. LUN 11 uses path B as the primary, LUN 12 back to path A, and so on.

It so happens that I’m using a DMX-4 for storage, and the policy we have is to use a fixed path policy. I realize that Round Robin would make this entire script moot, well, except for making sure that the PSP is correct. I also realize that PowerPath would be the ideal solution for EMC storage, but we don’t use it…that’s a story for another day.

This script is, admittedly, long…longer than I expected it to be. The original inspiration for this script came from Justin Emerson’s very functional and succinct script, however I was not satisfied with the way LUNs were balanced. His script queries the host for LUNs then sorts them by canonical name and round robins the paths based on the number of paths present for the first LUN.

This works well, so long as all the LUNs are present on all the hosts and they all have the same number of paths. I can only presume that he assumes that those cases have already been checked for, and fixed, prior to execution. I wanted to do that all in one script.

Additionally, and it’s rather petty, I wanted the LUNs to be balanced based off their LUN identifier rather than the canonical name…they don’t always follow the same order, and in the case of my hosts with two HBAs (and consequentially, two paths per LUN), I wanted all odd LUNs to use one path for the primary and all even LUNs to use the other. Justin’s script does an excellent job of ensuring that the paths are evenly distributed, as you will end up with the same number on each, but not in the pretty fashion I desired.

Also, thank you to Glenn, who helped me “powershellize” this script…my PowerShell looks and reads like Perl, and therefore doesn’t use a lot of the optimizations that PoSH brings…such as automatic parameter handling and other niceties.

So, without further ado…

[cmdletbinding(SupportsShouldProcess=$true)]
Param(
    # name of the cluster to modify
    [parameter(Mandatory=$False, ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
    [String]$clusterName = "Cluster_01"
,
    # the multipath policy to set for LUNs. be careful with RoundRobin though...it requires some extra params for the Set-ScsiLun command
    [parameter(Mandatory=$False, ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
    [ValidateSet("Fixed", "RoundRobin","MostRecentlyUsed")]
    [String]$multipathPolicy = "Fixed"
) 
Process
{
    Write-Verbose "Getting list of hosts to check/modify..."
    Try
    {
        $clusters = Get-View -ViewType ClusterComputeResource -Filter @{'Name'=$clusterName} -Property Name,Hosts
    }
    Catch
    {
        Write-Warning " cluster $clustername not found"
        return
    }
 
    Try
    {
        $hosts = Foreach ($c in $Clusters)
        {
            $c.Host | ForEach-Object { Get-VIObjectByVIView $_ }
        }
        $hosts = $hosts | Sort-Object -Name
    }
    Catch
    {
        Write-Warning "Error enumerating hosts in $clusterName "
        return
    }
 
 
    if ($hosts.Count -lt 1) 
    {
        Write-Warning "No hosts were found in cluster $clustername "
        return
    }
 
    # determine the LUNs available
    Write-verbose "Determing LUNs and paths based on $($hosts[0].Name)"
 
    # get the LUN order based on the first host
    $hostView = $hosts[0].ExtensionData
    $storageView = Get-View $hostView.ConfigManager.StorageSystem
    $hbas = $storageView.StorageDeviceInfo.ScsiTopology.Adapter | Where-Object { $_.Adapter -like "*FibreChannelHba*" }
 
    $lunList = @{}
    $lunCnames = @{}
 
    foreach ($hba in $hbas | Sort-Object -Property Key) {
        foreach ($target in $hba.Target | Sort-Object -Property Key) {
            foreach ($lun in $target.Lun | Sort-Object -Property Lun) {
 
                $l = New-Object PSObject -Property @{
                    Target = $target.Key.Split("-")[2]
                    Lun = $lun.Lun
                    ScsiLun = $lun.ScsiLun
                    CanonicalName = $(Get-ScsiLun -VmHost $hosts[0] -Key $lun.ScsiLun).CanonicalName
                    Name = $l.Target+":"+$l.Lun
                }
                Write-verbose "Found LUN $($l.Name)"
 
                if (!$lunCnames[$l.CanonicalName]) {
                    $lunCnames[$l.CanonicalName] = @()
                }
 
                if (!$lunList[$l.Target]) {
                    $lunList[$l.Target] = @()
                }
 
                $lunList[$l.Target] += $l
                $lunCnames[$l.CanonicalName] += $l
            }
        }
    }
 
    $lunOrder = @()
 
    # by sorting these two hashes before looping, we end up with the LUNs being in C:T:L order
    $lunList.GetEnumerator() | Sort-Object -Property Name | %{
        $_.Value | Sort-Object -Property Lun | %{
            if (!($lunOrder -contains $_.CanonicalName)) {
                # since it's sorted, we can simply add to an array and it will retain the order
                $lunOrder += $_.CanonicalName
            }
        }
    }
 
    $i = 0
    $lunFinal = @{}
    $lunPathCount = @{}
 
    # now that we have the LUN C:T:L order, we simply round robin the avail paths
    foreach ($cn in $lunOrder) {
        $lun = Get-ScsiLun -VmHost $hosts[0] -CanonicalName $cn
        $paths = Get-ScsiLunPath -ScsiLun $lun | Sort-Object -Property SanID
        $x = $i % $paths.Count
        $path = $paths[$x]
 
        # the hash will store them in random order, but we don't care cause the path to use
        # was determined from the ordered array object...basically, don't be concerned later
        # in execution when the report doesn't have each one alternating exactly
        $lunFinal.Add($cn, $path.SanID)
 
        $i++
 
        Write-Verbose "Using $($path.SanID) for LUN ${cn} ( $(($lunCnames[$cn] | Sort-Object -Property Name)[0].Name) )"
    }
 
    # let's check each host to make sure it has the LUNs we found before
    # and report on any new or missing LUNs
    Write-Verbose "Verifying LUNs are on all hosts..."
 
    foreach ($vmhost in $hosts) {
        Write-Verbose "Checking $($vmhost.Name)"
 
        $hostLuns = @()
        $hostView = $vmhost.ExtensionData
 
        # using the view is faster here and still has the info we want/need
        # the where clause on this line will need to be evaluated depending on the environment
        # currently I only have EMC arrays, which all LUNs that are *correctly* presented have 
        # an NAA name.  I don't think all arrays are that way...so this may need to be modified
        $hostView.Config.StorageDevice.ScsiLun | ?{ $_.CanonicalName -like "naa.*" } | %{
            $hostLuns += $_.CanonicalName
        }
 
        # check for each LUN, remove if found, report if not
        foreach ($cn in $lunFinal.GetEnumerator()) 
        {
                Write-verbose "Checking LUN $($cn.Key)"
 
            if ($hostLuns -contains $cn.Key) 
            {
                Write-Verbose "`tLUN found on host"
 
                # remove the LUN from the array so we know it was found
                $hostLuns = $hostLuns | ?{ $_ -ne $cn.Key }
 
                # check for correct number of paths
                $pathsOnFirstHost = $($lunCnames[$cn.Key]).Count
                $pathsOnThisHost = $($hostView.Config.StorageDevice.MultipathInfo.Lun | ?{ $_.Lun -eq $lunCnames[$cn.Key][0].ScsiLun }).Path.Count
 
                Write-Verbose "Found $pathsOnThisHost of $pathsOnFirstHost expected paths for LUN"
 
                if (! $pathsOnFirstHost -eq $pathsOnThisHost) 
                {
                    Write-warning "LUN $($cn.Key) does not have expected number of paths ( $pathsOnFirstHost expected, $pathsOnThisHost found )"
                }
             } 
             else 
             {
                Write-Warning "LUN $($cn.Key) was not found on this host"
            }
        }
 
        # anything left in the array must be unique to the host...e.g. local storage
        if ($hostLuns.Count -gt 0) {
            foreach ($cn in $hostLuns) {
                Write-verbose "LUN $($cn) was not found in initial scan and will be ignored!"
            }
        }
    }
 
    # now we set each host in the cluster to use the determined paths
    foreach ($vmhost in $hosts) 
    {
	    if ($vmhost.ConnectionState -ne "disconnected") {
		    Write-Verbose "Starting configuration of $($vmhost.Name)"
 
		    foreach ($lunCanonicalName in $lunFinal.GetEnumerator()) 
            {
			    $lun = $vmhost | Get-ScsiLun -CanonicalName $lunCanonicalName.Key
			    $path = Get-ScsiLunPath -ScsiLun $lun | ?{ $_.SanID -eq $lunCanonicalName.Value }
 
			    IF ($PSCmdlet.ShouldProcess($VMhost.Name, "Setting $($lun.CanonicalName) active/preferred path to $($path.SanID)"))
                {
				    $lun | Set-ScsiLun -MultipathPolicy $multipathPolicy -PreferredPath $path
 
				    # this sleep time may need to be adjusted for different arrays
				    Start-Sleep -Seconds 3
			    }
		    }
	    } else {
            Write-Warning "$($vmhost.Name) is disconnected from vCenter"
        }
    }
}