NetApp’s SnapManager for Virtual Infrastructure (SMVI) is a great product, but it’s messy. If it encounters the any error, it seemingly forgets to delete the virtual machine snapshots from the Virtual Infrastructure before dying.

To prevent many orphans (I’ve seen as many as 20 on a single virtual machine) from happening, I created a quick Nagios check that simply alerts when it sees them.

This script is very elementary. It very simply uses a regex to check for any snapshots that match the default SMVI naming convention. For each one it finds, a counter is incremented. If any are found, the script returns an error to Nagios, which causes an alert to be sent.

#!/usr/bin/perl -w
#
# check_vi_smvi_snapshots.pl - written by Andrew Sullivan, 2010-06-16
#
# Please report bugs and request improvements at http://get-admin.com/blog/?p=1059
#
# A simple script to look for snapshots that match the name pattern that smvi uses.
# We are merely pulling a list of all snapshots, searching for the string "smvi" in 
# the name, if it's found, we return a warning condition.  This could lead to a 
# "false" positive if it runs while a snapshot series is still ongoing, but since
# the smvi snaps should be very short lived the condidition will not last unless
# the snap is left.
#
# Example:
#   ./check_vi_smvi_snapshots.pl --server your.esx.host --username you --password secret
#
 
use strict;
use warnings;
 
use FindBin;
use lib "$FindBin::Bin/../";
 
use VMware::VIRuntime;
 
# substitute the location of your nagios perl library
use lib "/usr/lib64/nagios/plugins";
use utils qw(%ERRORS);
 
Opts::parse();
Opts::validate();
 
Util::connect();
 
main();
 
Util::disconnect();
 
sub main {
 
	# the number of smvi snapshots
	my $smviSnaps = 0;
 
	# for setting the type of exit we want
	my $exitCondition = "";
 
	# we need MORs for each of the VMs on the host
	my $VMs = Vim::find_entity_views( view_type => 'VirtualMachine' );
 
	foreach my $vm (@$VMs) {
		if ($vm->snapshot) {			
			foreach my $childSnapshot (@{$vm->snapshot->snapshotInfo->rootSnapshotList}) {
				$smviSnaps += getSnaps($childSnapshot);
			}
 
		} else {
			#print $vm->name . " has no snapshots\n";
		}
	}
 
	if ($smviSnaps > 0) {
		print "WARNING - " . $smviSnaps . " SMVI snapshots exist.\n";
		$exitCondition = "WARNING";
 
	} else {
		print "OK - No SMVI snapshots exist.\n";
		$exitCondition = "OK";
 
	}
 
	Util::disconnect();
	exit $ERRORS{ $exitCondition };
}
 
sub getSnaps {
	my ($snapshotTree) = @_;
	my $snapcount = 0;
 
	# uncomment for debugging
	#print "Found snap: " . $snapshotTree->{name} . "\n";
 
	if ( $snapshotTree->{name} =~ /smvi/ ) {
		$snapcount++;
	}
 
	if ($snapshotTree->childSnapshotList) {
		foreach my $childSnapshot (@{$snapshotTree->childSnapshotList}) {
			$snapcount += getSnaps($childSnapshot);
		}
	}
 
	return $snapcount;
}

I’ve set the check to execute once an hour in my environment, as I don’t feel that granularity finer than that is needed…an hour’s worth of change is ok for an SMVI snapshot for me.

I’ve been out of touch for most of this week, having only been able to be connected for an extended period of time today, and from somewhere out in left field I received an extremely surprising email from Mr. John Troyer…

I have been named a 2010 vExpert! Words can not describe how honored I am to receive this designation, I feel truly humbled by the others that have received the award and I can only hope that when I grow up I can be like them.

Thank you again to John Troyer and his team for this privilege!

Andrew

I can honestly say that I capitalized on a once in a lifetime opportunity.  For what ever reason Dev day was small this year.  There where only around 300 of us on the Developer track, and while the superstars of Virtualization were all looping through PTAP sessions I was attending small 15 to 1 labs with the likes of Steve Jin, Scott Herold, Carter Shanklin, LucD, and Cody Bunch…  You could say I learned a thing or two!

I started Monday with DS-13 it was suppose to be an Introduction to the vSphere Webservices SDK .  Unfortunately system errors prevented Steve from giving his full presentation!  My first lab I spent 45m trying to log into my virtual desktop.  It wasn’t a total lost though as I had Access to one of the authoritative sources on the VI API!  Shortly after Steve’s session I got a little side tracked, and picked my schedule back up with DS-16.

DS-16 Extending PowerCLI to Enterprise Applications with Virtualization EcoShell (VESI) presented by Scott Herold.  This session proved to be my favorite from Monday, and ran an hour long (in a good way)!  Good news, Scott and his team have done some fantastic work.  He is attempting to develop on demand.  Meaning as a demand for a feature/need starts to bubble up from the community either from the VESI forums or the usual places.  Scott fills that gap with a custom script extending the PowerCLI, or by modifying the user interface itself, extending VESI to better match the needs of the virtual administrator.  An example of the latter was on display where the VESI team has added the ability to transform any data set into rich charts. An important distinction with VESI is it is meant to enable the Virtualization Administrator NOT the VI Admin!  VESI will have full support for any Hypervisor/mgmt framework that the community has demand for.  It will also encompass any peripheral components of the virtual world.  Providing easy to use and context relevant access to any pain point whether it be storage, Network, AD… What ever the community needs!

The cynic out there will ask okay what does Vizioncore get out of this?  the answer, A single pane of glass that encompasses the entire virtualization ecosystem.  Oh yeah, and that pane of glass, it will one day serve as the front end for all of Vizioncores products!  The question was asked about pricing, and Scott insists that “VESI is and always will remain free”.  They need this framework for there own internal roadmap.  It’s extension to the community as a whole in my opinion will garnish them nothing but good will, and a built in user base.  Your probably asking yourself where’s the bad?

Politics… anyone from the PowerShell community will immediately recognize the VESI interface.  It’s our old friend PowerGUI, I asked Scott why something new, why not just build on top of PowerGUI.  His answer was speed, the PowerGUI team has a product roadmap, and there users need different things then Scotts.  He used the upcoming charts feature as an example.  It could take PowerGUI 18 months to get charts on there roadmap. PowerGUI is already hard at work putting out other fires.  By Scott forking PowerGUI he created a divisions but that division purchased an independent product roadmap.  It’s this roadmap that is enabling him to move with the Virtualization Community.  The sad part to me I don’t believe the division was truly necessary.  Why Scotts team couldn’t just develop those same features, injecting them into PowerGUI as needed, and thereby enhancing both products at once… that can only be political.  We all know how software works. There is no technical reason preventing this.  Alas while I think a best of both worlds super PowerGUI would have been better for everyone.  I for one am glad to have VESI in our tool belt. If your new to PowerShell or the PowerCLI check it out as Carter put it “VESI is the onramp to PowerCLI and PowerShell Scripting”… Couldn’t agree more!

Finally I ended Monday with a session on VIX.  while there is some really cool stuff coming in VIX there has been no change for the PowerShell community.  The latest version of VIX shipped just last week, and sadly 1.7 still offers no .Net/vi sdk interfaces. The .COM interface is critically crippled if you want to use it with vSphere, and overall your still forced to provide a username/Password to the guest OS.  Alas it’s not nearly as bad as I made it out to be!

The 1.7 release added full support for vSphere 4.0, and the VIX team is currently evaluating SSIP/Certification based authentication for the guest.  As for how it will ultimately be extended into powershell it looks like either a .net class, or by extending the vi api.  Either way will be a win for powershell as we can easily extend either into first-class cmdlets!  The use case for VIX is a bit nitch, but when you need it nothing else will do!

An interesting tidbit if you’re super security cautious you can disable VIX by adding

“Guest.Command.Enabled”=”False”

to either the VM or the host.  Be aware that this WILL break upgrading of VMware tools, and Guest customization as they both use vix as the underlying technology!

So that’s Developer day at VMworld all in all I had a blast, and met the superstars of the VI API/vSphere automation community.  The Food was 10x better then what they’re serving here at VMworld, and I get a free license of vSphere!  All for $249 USD, if you’re interested in more advanced automation at the vi api level I highly recommend developer day. 

VMware this was a win, win… let’s try and keep it for the future.

~Glenn

So there I was in-between sessions just after lunch at VMworld when someone on twitter mentioned a VCP button.  Being a big fan of badge bling I showed the post to Andrew, and asked some guys around us If any of them knew anything about it.  One had seen the button, but didn’t know where you got one or why.  This is where my Monday got a little sidetracked.  At first we headed down to the materials pickup where I caught John Troyer stealing buttons!  But they directed us to the customer service desk.

Andrew and I proceeded to the customer services desk to inquire, and were quickly informed that we needed to be VCP4.  Honestly, that should have been the end of it, but for whatever reason we decided to walk over and look for ourselves.  Next thing I know I’ve slapped down my AMEX and I’m sitting behind a person vue terminal!  Why is this significant?

I’ve never touched vSphere, no beta, no RC access… Nothing.. Heck I didn’t even review the configuration maximums before hand!  I took it totally off the cuff no prep at all, and  passed!  I have my VCP on VI 3, and I’ve kept up with vSphere via our wonderful bloggers.  In a month or so I should be the proud owner of a VCP 4 certification.  At this point I need throw a huge thank you over to John Troyer, because while I’ve never touched these features I have listened to extensive interviews/discussions covering every aspect or vSphere.  In many ways the VMware Communities Roundtable Podcast was my “what’s new what’s changed” course.  In addition  to that overview our community in particular Duncan. Scott, Boche, etc provided me with the real life reference implementations.

What I don’t have is the hands on experience, I lack the practical knowledge that comes from performing a real implementation (A.K.A. the part that matters).  So there you have it, everyone complained about the need for the class, and VMware listened.  My question to you is… should I be able pass that test.  Knowing the whole background would YOU accept my updated credentials?  Something to think about next time we start complaining about the need for a class.  Personally, I never would have upgraded if it weren’t for the grace period, so I for one am grateful, but is this what we really want?

Perhaps we can agree that I’m just that good..? I would say that I benefited from the world class engineers I studied by proxy.  Personally I believe all of the above to be true.  Perhaps we could agree that the VCP isn’t that big of a deal, that It’s not a VCDX…I passed the “I know what these do in theory” test. Not the “I know what these do in practice” test. we would then agree that we would test for those skills further up the stack. 

As a Microsoft Professional I’m very familiar with that type of system, but is that what we want? In theory it sounds like a better system, but there is no end once you start down that road.  Microsoft recently had to enact a master certification to find the real masters.  Think about that for a second… why do you need a test to prove ‘this guy really really knows what he’s doing’ shouldn’t the engineer certification handle that?   I don’t know the answer I just wanted to ask the question?

~Glenn

Configuring a PXE server to present the files and information needed for kickstarting your ESX hosts isn’t too difficult a task. It does require some basic unix/linux knowledge, but aside from that, not too bad. I use a CentOS virtual machine with just 256 MB of RAM (you’ll need at least 512 for a GUI, but one isn’t necessary) to act as the PXE server for my ESX hosts. This same virtual machine also serves as a management point, as it has access to the management lan and with the perl toolkit and rCLI installed I can automate much of the work I need to accomplish with the hosts.

I happen to segregate the different types of traffic on the ESX hosts onto different VLANs. This means management (COS/PXE), VMotion, IP Storage, and virtual machine traffic (usually several VLANs by itself) are all separate. It is important that the server (or virtual machine) that you are using is configured with at least one interface on the same VLAN/network that the ESX management network is on. That interface will also need to have a static IP address.

It is also important that DHCP is able to function on this network when the host is in a totally unconfigured state. This means if you are trunking to your ESX hosts you must have the native VLAN set to the same as your management VLAN and port channeling (802.11q / LACP) can not be turned on during the PXE process.

Continue Reading »

One of my long term tasks has been to figure out how to effectively virtualize our SPARC infrastructure. Turns out it isn’t as easy as I originally thought it would be, mostly because of Solaris 8 and the fact that I can’t get rid of it . Don’t ask me why (cause it irritates me to no end…) but I can not convince the stodgy Solaris 8 admins that their binaries will run in Solaris 10 without modification.

Continue Reading »

Blade servers are all the rage these days, but let’s jump back a bit and remember the original intent. Blades are a consolidation mechanism. The first blade servers consisted of more underpowered physical servers crammed into a smaller U (Rack Unit) footprint. Over time the chassis have become more redundant, and the blades have grown to equal their physical brethren, but the intent remains the same! Sounds a lot like the mission statement for virtualization doesn’t it? I have issues with Blade servers and virtualization:

1. Most Datacenters don’t have enough PSC (Power, Space, and Cooling) to support blades.
2. If you’re using virtualization then you already have a consolidation mechanism.
3. Your upgrade path is fairly static, and exorbitantly expensive.
4. Losing a blade center is too destructive to the IT organization.

Continue Reading »

Disaster mitigation and recovery is a subject that is near and dear to my heart. As Glenn can attest, I am extremely conservative when it comes to datacenter design…I am a firm believer in redundant redundancy.

I have almost 100% blade centers, for better or worse, in my virtualization infrastructure. Let’s assume I have four brand new blade chassis, each with 14 blades (care to guess who they’re from?), for a total of 56 blades to assimilate. Rather than assign the blades to clusters sequentially, I wanted to ensure that the loss of a blade chassis was mitigated as much as possible. To do this, as hardware is assigned to the virtual clusters and datacenters it is taken in stripes across the blade chassis.

Continue Reading »

I love my Job!!! I’m currently involved in our virtualization initiative… Were still in the crawling stage eight ESX hosts in two DRS clusters.  In the traditional admin mantra we’ve rebuilt our whole infrastructure four or five times.  After reading white papers till 2am, and tearing my hair out over configuration mistakes (Google ESX partition offset). My boss decided to just send us to VMworld!

Snoopy Dance!!!

~Glenn