get-admin

From time to time I try and update all bios/firmware/etc. While VMotion makes this all but a cinch, evacuating every host in succession puts quite a strain on DRS.   Therefore,  I *try* and patch everything when installing uudates.  I’ve developed my own little technique where I let VUM do it’s thing.  Catching the Host in the middle of the reboot, and update everything else.  While the actual update methodology changes from OEM to OEM.  I use the same simple little script for vSphere/ESX.  Basically, when I see the remediation task start for a host I kick this function off, “watching” that host, and walk away. Then when vCenter registers a VMHost reboot on the Host this little guy let’s me know.

Function WatchVMHostReboot {
    param(
        [VMware.VimAutomation.Client20.VMHostImpl]
        $VMhost
    )
    $running = $false
    $start = ("{0:MM/dd/yyyy}" -f [datetime]::now)
    while (-Not $running){
        $events = Get-VIEvent -Entity $VMHost `
            -Username com.vmware.vcIntegrity `
            -Types info `
            -Start $start |
                Where-Object {$_.fullFormattedMessage -match "reboot"}
        if ($events) {
            Write-Host "reboot on $($vmhost.Name)`a`a`a"
            $running = $True
        }
    }
}

In case you didn’t already know `a will cause your motherboard to beep.  That loud annoying  “I can hear it two cubicles over” beep!

~Glenn

My Virtual Infrastructure was recently audited.  As part of my preparation for said audit I needed to verify that several extra configuration Parameters were set on every VM. Nothing ground breaking, this has all been covered here, and here. So why the repost, well I’m obsessed with scaling! I don’t like doing anything that I can’t use to the nth degree. Having said that I found two simple tweaks that dramatically increased the performance of these scripts.

If you ever find yourself using where-object move back up the pipeline… can you use a filter instead? Here I dramatically improved performance by leveraging the built-in filter capabilities of Get-View. I was also able to crank it up by simply switching from the ReconfigVM method to the ReconfigVM_Task method. Unless your performing some serial action, always, always use the task method. Offloading the babysitting to vCenter just makes sense! Finally, I loath text files, especially when they create a needless dependencies. Here I use a simple hashtable to embed my configuration in the script it self.

I successfully used this script to update over 500 vm’s in less than 4min!  Now that is what I call scale!  I know the security experts our there would argue that this is meaningless, b/c of this or that… all I know is I passes my audit with flying colors (didn’t have one ding on a VM’s configuration).

$ExtraOptions = @{
    "isolation.tools.copy.disable"="true";
    "isolation.tools.paste.disable"="true";
    "isolation.tools.diskShrink.disable"="true";
    "isolation.tools.diskWiper.disable"="true";
    "isolation.tools.connectable.disable"="true";
    "isolation.tools.setGUIOptions.Enable"="false";
    "log.keepOld"="10";
    "log.rotateSize"="100000"
}
 
# build our configspec using the hashtable from above.  I prefer this
# method over the use of files b/c it has one less needless dependency.
$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec
# note we have to call the GetEnumerator before we can iterate through
Foreach ($Option in $ExtraOptions.GetEnumerator()) {
    $OptionValue = New-Object VMware.Vim.optionvalue
    $OptionValue.Key = $Option.Key
    $OptionValue.Value = $Option.Value
    $vmConfigSpec.extraconfig += $OptionValue
}
# Get all vm's not including templates
$VMs = Get-View -ViewType VirtualMachine -Property Name -Filter @{"Config.Template"="false"}
 
# Do it!
foreach($vm in $vms){
    $vm.ReconfigVM_Task($vmConfigSpec)
}

~Glenn

I can honestly say that I capitalized on a once in a lifetime opportunity.  For what ever reason Dev day was small this year.  There where only around 300 of us on the Developer track, and while the superstars of Virtualization were all looping through PTAP sessions I was attending small 15 to 1 labs with the likes of Steve Jin, Scott Herold, Carter Shanklin, LucD, and Cody Bunch…  You could say I learned a thing or two!

I started Monday with DS-13 it was suppose to be an Introduction to the vSphere Webservices SDK .  Unfortunately system errors prevented Steve from giving his full presentation!  My first lab I spent 45m trying to log into my virtual desktop.  It wasn’t a total lost though as I had Access to one of the authoritative sources on the VI API!  Shortly after Steve’s session I got a little side tracked, and picked my schedule back up with DS-16.

DS-16 Extending PowerCLI to Enterprise Applications with Virtualization EcoShell (VESI) presented by Scott Herold.  This session proved to be my favorite from Monday, and ran an hour long (in a good way)!  Good news, Scott and his team have done some fantastic work.  He is attempting to develop on demand.  Meaning as a demand for a feature/need starts to bubble up from the community either from the VESI forums or the usual places.  Scott fills that gap with a custom script extending the PowerCLI, or by modifying the user interface itself, extending VESI to better match the needs of the virtual administrator.  An example of the latter was on display where the VESI team has added the ability to transform any data set into rich charts. An important distinction with VESI is it is meant to enable the Virtualization Administrator NOT the VI Admin!  VESI will have full support for any Hypervisor/mgmt framework that the community has demand for.  It will also encompass any peripheral components of the virtual world.  Providing easy to use and context relevant access to any pain point whether it be storage, Network, AD… What ever the community needs!

The cynic out there will ask okay what does Vizioncore get out of this?  the answer, A single pane of glass that encompasses the entire virtualization ecosystem.  Oh yeah, and that pane of glass, it will one day serve as the front end for all of Vizioncores products!  The question was asked about pricing, and Scott insists that “VESI is and always will remain free”.  They need this framework for there own internal roadmap.  It’s extension to the community as a whole in my opinion will garnish them nothing but good will, and a built in user base.  Your probably asking yourself where’s the bad?

Politics… anyone from the PowerShell community will immediately recognize the VESI interface.  It’s our old friend PowerGUI, I asked Scott why something new, why not just build on top of PowerGUI.  His answer was speed, the PowerGUI team has a product roadmap, and there users need different things then Scotts.  He used the upcoming charts feature as an example.  It could take PowerGUI 18 months to get charts on there roadmap. PowerGUI is already hard at work putting out other fires.  By Scott forking PowerGUI he created a divisions but that division purchased an independent product roadmap.  It’s this roadmap that is enabling him to move with the Virtualization Community.  The sad part to me I don’t believe the division was truly necessary.  Why Scotts team couldn’t just develop those same features, injecting them into PowerGUI as needed, and thereby enhancing both products at once… that can only be political.  We all know how software works. There is no technical reason preventing this.  Alas while I think a best of both worlds super PowerGUI would have been better for everyone.  I for one am glad to have VESI in our tool belt. If your new to PowerShell or the PowerCLI check it out as Carter put it “VESI is the onramp to PowerCLI and PowerShell Scripting”… Couldn’t agree more!

Finally I ended Monday with a session on VIX.  while there is some really cool stuff coming in VIX there has been no change for the PowerShell community.  The latest version of VIX shipped just last week, and sadly 1.7 still offers no .Net/vi sdk interfaces. The .COM interface is critically crippled if you want to use it with vSphere, and overall your still forced to provide a username/Password to the guest OS.  Alas it’s not nearly as bad as I made it out to be!

The 1.7 release added full support for vSphere 4.0, and the VIX team is currently evaluating SSIP/Certification based authentication for the guest.  As for how it will ultimately be extended into powershell it looks like either a .net class, or by extending the vi api.  Either way will be a win for powershell as we can easily extend either into first-class cmdlets!  The use case for VIX is a bit nitch, but when you need it nothing else will do!

An interesting tidbit if you’re super security cautious you can disable VIX by adding

“Guest.Command.Enabled”=”False”

to either the VM or the host.  Be aware that this WILL break upgrading of VMware tools, and Guest customization as they both use vix as the underlying technology!

So that’s Developer day at VMworld all in all I had a blast, and met the superstars of the VI API/vSphere automation community.  The Food was 10x better then what they’re serving here at VMworld, and I get a free license of vSphere!  All for $249 USD, if you’re interested in more advanced automation at the vi api level I highly recommend developer day. 

VMware this was a win, win… let’s try and keep it for the future.

~Glenn

As part of a larger migration I was recently given the task to copy all resource pools into new EVC enabled clusters. An already monotonous task was made worse by our use of resource pools for delegation. While it took a couple days to work out a recursion bug. The end product worked flawlessly, and successfully reproduced almost 100 resource pools across three clusters in less than 5min.

I mention the time lost because I truly believe you should “try” to script everything! In this case I lost some time, but I gained accuracy, and the ability to scale to the nth degree!

Continue Reading »

At seemingly random intervals I get requests from management to report every Virtual Machine that was added/removed in the last x days. This has traditionally been a manual process, and one that I loath.

While deploying a VM this morning I set up my normal tell me when your done one-liner…

Get-Task|?{$_.Name -eq "CloneVM_Task"}|Wait-Task; `a`a`a

As I was typing I wondered how long a task’s history was kept, and if I could query it via the SDK? Low and behold not only can I but it’s too easy!

#requires -pssnapin VMware.Vimautomation.Core
Param(
    [int]$LastDays
)
Process
{
    $EventFilterSpecByTime = New-Object VMware.Vim.EventFilterSpecByTime
    If ($LastDays)
    {
        $EventFilterSpecByTime.BeginTime = (get-date).AddDays(-$($LastDays))
    }
    $EventFilterSpec = New-Object VMware.Vim.EventFilterSpec
    $EventFilterSpec.Time = $EventFilterSpecByTime
    $EventFilterSpec.DisableFullMessage = $False
    $EventFilterSpec.Type = "VmCreatedEvent","VmDeployedEvent","VmClonedEvent","VmDiscoveredEvent","VmRegisteredEvent"
    $EventManager = Get-View EventManager
    $NewVmTasks = $EventManager.QueryEvents($EventFilterSpec)
 
    Foreach ($Task in $NewVmTasks)
    {
        # If VM was deployed from a template then record which template.
        If ($Task.Template -and ($Task.SrcTemplate.Vm))
        {
 
            $srcTemplate = (Get-View $Task.SrcTemplate.Vm -Property name).Name
        }
        Else
        {
            $srcTemplate = $null
        }
        write-output ""|Select-Object @{
                Name="Name"
                Expression={$Task.Vm.name}
            }, @{
                Name="Created"
                Expression={$Task.CreatedTime}
            }, @{
                Name="UserName"
                Expression={$Task.UserName}
            }, @{        
                Name="Type"
                Expression={$Task.gettype().name}
            }, @{
                Name="Template"
                Expression={$srcTemplate}
            }
    } 
}

Enjoy,
~Glenn

UPDATE:  This Functionality has been completely replaced with the Update-Tools cmdlet!

This morning I was updating a smaller farm to 3.5u4. 15 minuets into the tools upgrade process I had enough… Enter PowerShell, I whipped up a quick script that will update All Windows VM’s to the latest version of VMware Tools.

$starttime = get-date
#requires -version 2
$GuestCred = Get-Credential
$OldTools = Get-View -ViewType "VirtualMachine" `
    -Property Guest,name `
    -filter @{
        "Guest.GuestFamily"="windowsGuest";
        "Guest.ToolsStatus"="ToolsOld";
        "Guest.GuestState"="running"
    }
 
Foreach ($VM in $OldTools) {
    # Mount the tools CD
    $VM.MountToolsInstaller()
 
    # Get the drive letter for the CD Drive tools is mounted in
    $DrvLetter = Get-WmiObject Win32_CDROMDrive -ComputerName $VM.name -Credential $GuestCred |
        Where-Object {$_.VolumeName -match "VMware Tools"} |
        Select-Object -ExpandProperty Drive
 
    # our update cmd
    $cmd = "$($DrvLetter)setup.exe /S /v`"/qn REBOOT=ReallySuppress REINSTALLMODE=vomus REINSTALL=ALL`""
 
    # Create a new process on the VM, and execute setup
    $go = Invoke-WMIMethod -path win32_process `
        -Name Create `
        -Credential $GuestCred `
        -ComputerName $VM.Name `
        -ArgumentList $cmd
 
    # If we sucessfully spawned an upgrade wait for it to finish
    IF ($go.ReturnValue -eq 0)
    {
        $i=0
        While ($VM.Guest.ToolsStatus -ne 'toolsOk')
        {
            Start-Sleep -Seconds 1
            $VM.UpdateViewData("Guest")
            if ($I -gt 120)
            {
                Write-Warning "$($VM.name) appears to have hung, please investigate!"
                break
            }
            $i++
        }
        $vm.UnmountToolsInstaller()
        Write-verbose "$($VM.Name) Successfully updated"
    }
    else
    {
        Write-Warning "error $(go.ReturnValue) triggering tools install on $($VM.name). "
        $vm.UnmountToolsInstaller()
    }
}
$processed = $oldTools.count - (Get-View -ViewType "VirtualMachine" `
    -Property Guest,name `
    -filter @{
        "Guest.GuestFamily"="windowsGuest";
        "Guest.ToolsStatus"="ToolsOld";
        "Guest.GuestState"="running"
    }).count
$ts = New-TimeSpan -Start $starttime
write-host ("{0} out of {1} vm's were updated in {2}.{3} Min" -f $processed, $oldTools.count, $ts.Minutes,$ts.Seconds)

Using this script I updated 50 Vm’s in 40min. There is alot of room for improvement here. I opted for the slow and steady pace, but larger environments could easily tune this to 100 at a time.

~Glenn

P.S. I started with Invoke-VMScript but only half of the vm’s had PowerShell installed, and I needed something that would hit them all.

Everyone knows how to use the Get-View cmdlet to retrieve the full managed object from a VI implementation (Impl) object, but did you know you can skip the Impl entirely? For instance have you ever needed the following?

$VM = Get-VM | Get-View

If so you may want to try:

$VM = Get-View -ViewType "VirtualMachine"

I’ve found the latter to be around 1200% faster!  Additionally, by selecting the properties of interest at run time  an additional performance boost can be obtained.  For example suppose you just wanted the Name of every ESX host in vCenter.

$VMHostNames= Get-View -ViewType "HostSystem" -Property Name |
    Select-Object -ExpandProperty name

is 300% faster then the traditional

$VMHostNames= Get-VMHost | Select-Object -ExpandProperty name

When you really need to optimize you VI Scripts just add some filters. For instance a 400% performance boost can be obtained by replacing,

$VM = Get-VM -Name "XP"

with

$VM = Get-View -ViewType "VirtualMachine" -Filter @{"Name"="XP"}

but what if you need the Impl object?  It’s still 150% faster to convert back from the managed object that to retrieve then to get the impl in the first place.

$VM = Get-View -ViewType "VirtualMachine" -Filter @{"Name"="XP"} |  Get-VIObjectByVIView

Finally the grand pooh-bah of hidden potential… nested parameters within filters!  Say you want to locate all windows vm’s that have out of date tools.

traditionally we would approach this with something like,

Get-VM |
    Where-Object {
        $_.PowerState -eq "PoweredON" `
        -and `
        $_.Guest.OSFullName -match "Windows"
    } |
    Get-View |
    where { $_.guest.toolsstatus -match "toolsOld" }

Wait for it….

Get-View -ViewType "VirtualMachine" `
    -filter @{
        "Guest.GuestFamily"="windowsGuest";
        "Guest.ToolsStatus"="ToolsOld";
        "Guest.GuestState"="running"
    }

Over 800% faster and WAY less code! Although using get-view to it’s full potential can seriously increase your scripts performance I would leave it in your scripts.   The cmdlets provided in the PowerCLI are built around an interactive experience, the SDK is NOT!  While it may be faster to get all vm’s with get-view. The information returned from Get-VM is more useful when working from the cmdline.

Happy Scripting,
~Glenn

Bonus:  The function I was writing when I discovered this enhanced functionality… that’s right it was *hidden* in the help.

Function Get-OldTools {
    Get-View -ViewType "VirtualMachine" `
        -property Guest, Name `
        -filter @{
            "Guest.GuestFamily"="windowsGuest";
            "Guest.ToolsStatus"="ToolsOld";
            "Guest.GuestState"="running"
        } |
        Select-Object -Property Name, @{
            Name="ToolsVersion"
            Expression={$_.Guest.ToolsVersion}
        }, MoRef | 
        Sort-Object ToolsVersion
}