Being, first and foremost, lazy and getting my paychecks for being a system administrator, I felt that the amount of work involved in loading ESXi 4.0 on my blades was entirely too much. I have well over 100 blades, each one needing to have vSphere loaded onto it, configured, and added to vCenter. Even using the directions scattered across the internet about reducing the amount of effort involved in loading vSphere was too much for me.

Others have documented how to PXE boot ESXi elsewhere on the internet, however I wasn’t interested in having a “stateless” install…I merely wanted to automate installing ESXi to the local hard drive. My blades have a single hard drive, a single generation one SSD or two SAS drives in a RAID 1 depending on the vendor, and I simply want the installer to always install to that drive without bothering me. Loading from the “remote media” functionality of the DRAC/iLO for the blades takes forever, so I wanted to be able to install using PXE and push the media over that medium.

So, having been a developer for several years I decided to dive further into the the install process than others had detailed. Turns out that eliminating all input from an administrator to load the operating system was pretty simple.

The end result is that I am able to power on a blade, hit F12 to have it PXE boot and walk away. Some time later, we can use PowerShell and the PowerCLI to find the hosts (they will be somewhere in the DHCP scope of the provisioning LAN), give them a permanent IP and hostname, then configure them and add them to vCenter. By using PXE and the interactionless (yes, I did make up that word) install, I cut the time to load ESXi from about 45 minutes (using the remote media function takes FOREVER!) to less than 10.

Continue Reading »

As part of a larger migration I was recently given the task to copy all resource pools into new EVC enabled clusters. An already monotonous task was made worse by our use of resource pools for delegation. While it took a couple days to work out a recursion bug. The end product worked flawlessly, and successfully reproduced almost 100 resource pools across three clusters in less than 5min.

I mention the time lost because I truly believe you should “try” to script everything! In this case I lost some time, but I gained accuracy, and the ability to scale to the nth degree!

Continue Reading »

Another itch to scratch: which vSwitch is a pNIC connected to? To solve this simple problem I created a quick perl script…

This script also lets me see the driver in use, connection speed and duplex setting, and the MAC address of the pNIC.

# Sample output:
Adaptor (Driver)        Speed (Duplex)          MAC                     vSwitch
----------------        --------------          ---                     -------
vmnic1 (bnx2)           1000 (Full)             00:00:00:00:00:00       vSwitch0
vmnic0 (tg3)            1000 (Full)             00:00:00:00:00:00       vSwitch0
vmnic3 (tg3)            1000 (Full)             00:00:00:00:00:00       vSwitch1
vmnic2 (bnx2)           1000 (Full)             00:00:00:00:00:00       vSwitch1

Continue Reading »

I wanted get a list of port groups and their type (kernel, console, virtual machine) from a series of hosts, however the only thing I could find that was even close was a POSH script in the VMTN forums that was posted by LucD.

Using that script for inspiration, I essentially duplicated the functionality, but using the perl toolkit. This script gives me an easy to read (and parse…) list of portgroups, the vSwitch they belong to, and the type.

Continue Reading »

Configuring a PXE server to present the files and information needed for kickstarting your ESX hosts isn’t too difficult a task. It does require some basic unix/linux knowledge, but aside from that, not too bad. I use a CentOS virtual machine with just 256 MB of RAM (you’ll need at least 512 for a GUI, but one isn’t necessary) to act as the PXE server for my ESX hosts. This same virtual machine also serves as a management point, as it has access to the management lan and with the perl toolkit and rCLI installed I can automate much of the work I need to accomplish with the hosts.

I happen to segregate the different types of traffic on the ESX hosts onto different VLANs. This means management (COS/PXE), VMotion, IP Storage, and virtual machine traffic (usually several VLANs by itself) are all separate. It is important that the server (or virtual machine) that you are using is configured with at least one interface on the same VLAN/network that the ESX management network is on. That interface will also need to have a static IP address.

It is also important that DHCP is able to function on this network when the host is in a totally unconfigured state. This means if you are trunking to your ESX hosts you must have the native VLAN set to the same as your management VLAN and port channeling (802.11q / LACP) can not be turned on during the PXE process.

Continue Reading »

I dislike having to SSH into each host I am responsible for, and I detest having to enable SSH on ESXi (there should be NO reason for me to have to enable it). Because it’s difficult to script applying the NFS snapshot fix to a lot of hosts using the SSH method (and impossible if you don’t enable it on ESXi), I fooled around with the vifs.pl command that is provided with the rCLI.

I discovered that I can pull certain configuration files for the host using the command, modify them, then replace the configuration file…all without having to SSH to the host! vm-help.com has an excellent list of files available using this method.

All of the commands I use in the below script are available when the rCLI is installed (the rCLI also installs the perl toolkit, so all those “sample” scripts are available to us).

My windows scripting skills are non-existent, so I don’t know how to write a wrapper around the rCLI commands like I can with bash, but these same commands will work if you are using rCLI installed on Windows.

Continue Reading »

I had the need to change the configuration of my ESX hosts so that the virtual switches had a single active and single standby adapter assigned to them. The reason for the need is rather irritating (the IBM I/O modules that we have in these particular blade centers are not really designed to handle a high amount of traffic), and it was causing some issues during vMotions.

This script allows me set the vmnics assigned to a vswitch to the desired active/standby configuration, and additionally allows me to set the port group’s vmnic active/standby policy. In my setup, I use two vSwitches, one for primary COS, vMotion and IP storage, and a second vSwitch for the virtual machines and secondary COS, each vSwitch has two NICs assigned (remember, they’re blade centers…limited network connectivity). In order to avoid vMotion taking all the bandwidth for storage I wanted to separate their traffic onto different NICs, but still provide redundancy.

The way that I accomplish this is by making the default for the vSwitch have, for example, vmnic0 active and vmnic2 standby. I then adjust the vMotion port group so that it has the opposite (vmnic2 active and vmnic0 standby). Redundancy is still present in the event of a NIC failure, but under normal circumstances, the traffic is separate.

Continue Reading »

Edit 2009-02-09: I’ve updated the script slightly to reflect some errors that were occurring because I suck at regex.

I wanted a quick way of showing all snapshots for the VMs in vCenter using perl, so I spent a few minutes on this script. There are a lot of scripts for creating and deleting snapshots (and a couple to show them) using powershell, but not many using perl. I’m a *nix guy, so I wasn’t really interested in including the ability to send a mail to yourself or others in the script (just create a bash wrapper and use mail/mailx with a cron job) which saved me some time.

Well, after writing this script, I discovered that VMware included this functionality in their snapshotmanager.pl sample script, which is included with the Perl Toolkit.

Without further rambling by me, some perl….

Continue Reading »

For some reason, some of my servers have been having trouble with AD authentication. The symptoms were: when logging into the console using an AD enabled account (i.e. an account that should be authenticated by AD), despite providing the correct password the system will return password incorrect. Upon providing the username again, the user is immediately granted access without having to provide a password.

I’m not sure about the provide-once-denied-provide-name-authorized behavior, but after some testing I discovered that the reason that it’s not letting the user in the first time is because the lines for pam_unix and pam_kerberos (pam_krb5 to be exact) in /etc/pam.d/system-auth are reversed. It should have pam_unix before pam_kerberos. I have no clue what is causing these lines to be reversed in the configuration file. I’m configuring all of the authentication and security measures using the esxcfg-... commands so it seems weird that starting with the blade servers this behavior has been exhibited.

Anyway, the fix files: one bash, one awk…

Continue Reading »

After learning about the whole NFS fiasco I decided to create a quick bash script that would be part of my post kickstart process to apply the recommendations from NetApp as well as the fix required by VMware.

mmmmm, BASHy goodness….

Continue Reading »