Really NetApp?!? You didn’t use your own SDK?

So, this post irked me. Not because of the poster or his post (honest Andy, if you ever read this, I have nothing against you or your post! I’m happy to see another VMware/NetApp blogger!), but because of the script he referenced and the problem encountered. He has a good solution, but the problem shouldn’t exist.

You see, I hate RSH. I don’t know why (well, it is quite insecure, and it can require some configuration), but I hate it. SSH is only marginally better in this case…sure it’s secure, but you have to auth each time, and if you don’t (ssh keys), well, it’s only a little better than RSH (comms are encrypted, but compromise of a single account can lead to bad things on many hosts). The script that is referenced, one that NetApp recommends that admins use to verify that their aggregates have enough free space to hold the metadata for the volumes in OnTAP 7.3 (the metadata gets moved from the volumes to the aggregate in 7.3), uses RSH to execute commands that are then parsed in a somewhat rudimentary way to get information.

Sure, it’s effective, but it’s far from graceful…especially when you have a perfectly good and effective SDK at your disposal.

I was kind of bored, so I decided to rewrite the script using the SDK. This is the end result. It reports the same data, but uses the SDK to gather all of the necessary information to make a determination for the user. The new script is significantly shorter (10KB vs 25KB, 380 lines vs 980), and it requires only one login.

Thanks to NetApp for providing their SDK, and I hope that no one over there minds me refactoring…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
#!/usr/bin/perl -w
#
# na-aggrSpaceCheck.pl - written by Andrew Sullivan, 2009-06-26
#
# Please report bugs and request improvements at http://get-admin.com/blog/?p=773
#
# My intrepretation of NetApp's aggrSpaceCheck.pl.  This script will
# determine if an aggregate has enough free space to hold the metadata
# for the volumes contained in it when migrating to OnTAP 7.3.
#
# The original script: http://now.netapp.com/NOW/download/tools/aggrSpaceCheck/
#
# I apologize in advance is my refactor of the above script offends anyone
# at NetApp :)
#
# This script uses the OnTAP SDK, available here: 
#   http://communities.netapp.com/docs/DOC-1110
#
 
# I placed the NetApp perl modules into my perl lib directory, however,
# if you haven't done this, you will probably need to specify where they
# are using a lib declaration
#use lib "./NetApp";
 
use NaServer;
use NaElement;
 
use Data::Dumper;
use Getopt::Long qw(:config no_ignore_case);
 
main( );
 
sub main {
	my $opts = parse_options();
	my $server = get_filer( $opts->{ 'hostname' }, $opts->{ 'username' }, $opts->{ 'password' } );
 
	my $aggregates = get_aggregates( $server );
 
	print "found the following aggregates: " . "\n" if $opts->{'verbose'};
	print Dumper($aggregates) if $opts->{'verbose'};
 
	foreach my $key (keys %$aggregates) {
		my $aggr = $aggregates->{ $key };
		my $tax = 0;
 
		# metadata is already in the aggregate if it's a trad vol, so we don't
		# need to check them
		if ($aggr->{'type'} ne "trad") {
 
			foreach my $vol ( @{ $aggr->{'volumes'} } ) {
				if ($vol->{'online'}) {
					if ($vol->{'flexcache'}) {
						# .2%
						$tax += $vol->{'size'} * .002;
					} else {
						# .5%
						$tax += $vol->{'size'} * .005;
					}
				} else {
					print "FlexVol " . $vol->{'name'} . " is not online! \n";
				}
			}
 
			print $key . " needs " . get_printable_size($tax) . ". \n";
 
			if ($tax > $aggr->{'size-available'}) {
				print "The aggregate does not have enough available space! You need \n";
				print get_printable_size($tax - $aggr->{'size-available'}) . " additional capacity in the aggregate.";
			} else {
				print "The aggregate has " . get_printable_size($aggr->{'size-available'}) . " free.";
			}
 
			print "\n\n";
 
		}
	}
}
 
#
# Collect aggregate information for $server
#
sub get_aggregates {
	my $server = shift;
 
	my $request = NaElement->new('aggr-list-info');
	my $result = $server->invoke_elem($request);
 
	my $return = {};
 
	foreach ( $result->child_get('aggregates')->children_get() ) {
		my $name = $_->child_get_string('name');
 
		print "parsing aggregate " . $name . "\n" if $opts->{'verbose'};
 
		my $aggregate = {};
 
		# either trad or aggr: trad = traditional volume, aggr = contains flexvols
		$aggregate->{'type'} = $_->child_get_string('type');
 
		# the size reported as available, duh...
		$aggregate->{'size-available'} = $_->child_get_int('size-available');
 
		# the name is the only thing returned, so we must do another query
		my @volumes = ();
 
		foreach my $volume ($_->child_get('volumes')->children_get()) {
			push(@volumes, get_volume( $server, $volume->child_get_string('name')));
		}
 
		$aggregate->{'volumes'} = \@volumes;
 
		$return->{$name} = $aggregate;
 
	}
 
	return $return;
 
}
 
#
# gets the relavent volume information
#
sub get_volume {
	my ($server, $volume_name) = @_;
 
	print "\tparsing volume " . $volume_name . "\n" if $opts->{'verbose'};
 
	my $return = {};
 
	my $request = NaElement->new('volume-list-info');
	$request->child_add_string('volume', $volume_name);
 
	my $result = $server->invoke_elem($request);
 
	my $vol = $result->child_get('volumes')->child_get('volume-info');
 
	# some basic info about the volume
	$return->{'name'} = $vol->child_get_string('name');
 
	# is the volume online?  There are several other values possible, none of them
	# has the same meaning as online, so they are ignored
	$return->{'online'} = ( $vol->child_get_string('state') eq "online" ? 1 : 0 );
 
	print "\t\tvolume is " . ($return->{'online'} ? "online" : "offline") . "\n" if $opts->{'verbose'};
 
	# if the volume isn't online, we don't need to do any other work
	if ( $return->{'online'} ) {
		# if this is a flexcache volume, this object will exist in the returned
		# values.  Otherwise, it does not exist.
		$return->{'flexcache'} = ( $vol->child_get_string('remote-location') ? 1 : 0 );
 
		print "\t\tvolume " . ($return->{'flexcache'} ? "is" : "is not") . " flexcache\n" if $opts->{'verbose'};
 
		# in order to determine how much metadata the volume could/will have we need
		# to get the container size as reported by wv_fsinfo_blks_total, and we need
		# to calculate an adjusted container size based on the nominal size and a 
		# multiplier based on the type of volume it is.  The larger of those two 
		# values is then taken and used to determine the amount of metadata
 
		# nominal size is simply the size in bytes reported by the volume itself
		$return->{'nominalsize'} = $vol->child_get_int('size-total');
 
		print "\t\tvolume nominal size is " . $return->{'nominalsize'} . "\n" if $opts->{'verbose'};
 
		# blocks is the number of blocks that is reported from the perf object 
		# wv_fsinfo_blks_total counter.  that number is then multiplied by 4096 to
		# determine the number of bytes in the volume
		$return->{'containersize'} = get_vol_containersize( $server, $volume_name );
 
		print "\t\tvolume container size is " . $return->{'containersize'} . "\n" if $opts->{'verbose'};
 
		# we need to query the volume-size object to determine if it's a fixed size.  
		# to determine if it's a snapmirror, use the snapmirror-get-volume-status object.  
		my $is_fixedsize = get_vol_fixedsize( $server, $volume_name );
		my $is_snapmirror = get_vol_snapmirror( $server, $volume_name );
 
		print "\t\tvolume " . ($is_fixedsize ? "is" : "is not") . " fixed size\n" if $opts->{'verbose'};
		print "\t\tvolume " . ($is_snapmirror ? "is" : "is not") . " a snapmirror\n" if $opts->{'verbose'};
 
		if ($is_fixedsize || $is_snapmirror) {
			# this comes directly from the source script
			# four gibibytes...
			my $max_nvram_size_bytes = 1024 * 1024 * 1024 * 4;
 
			if ( 2 * $return->{'nominalsize'} < $max_nvram_size_bytes ) {
				$return->{'adjcontainersize'} = 2 * $return->{'nominalsize'};
			} elsif ( 1.1 * $return->{'nominalsize'} < $max_nvram_size_bytes ) {
				$return->{'adjcontainersize'} = $max_nvram_size_bytes;
			} else {
				$return->{'adjcontainersize'} = 1.1 * $return->{'nominalsize'};
			}
 
			print "\t\tvolume adj container size is " . $return->{'adjcontainersize'} . "\n" if $opts->{'verbose'};
 
			# if the adjusted size is greater than then reported size, then we use the
			# adjusted size for the volume
			if ( $return->{'adjcontainersize'} > $return->{'containersize'} ) {
				print "\t\tvolume adj size is larger than contsize: setting to adj size\n" if $opts->{'verbose'};
				$return->{'size'} = $return->{'adjcontainersize'};
			} else {
				print "\t\tvolume adj size is NOT larger than contsize: setting to adj size\n" if $opts->{'verbose'};
				$return->{'size'} = $return->{'containersize'};
			}
		} else {
			print "\t\tNot snapmirror or fixed, size is containersize\n" if $opts->{'verbose'};
			$return->{'size'} = $return->{'containersize'};
		}
	}
 
	return $return;
}
 
#
# Returns the number of bytes reported by WAFL that the continer occupies
#
sub get_vol_containersize {
	my ($server, $volume_name) = @_;
 
	# since the source script uses the number of blocks reported by wv_fsinfo_blks_total, we have to 
	# query the perf object to get that value
	my $request = NaElement->new('perf-object-get-instances');
	$request->child_add_string('objectname', 'volume');
 
	my $counters = NaElement->new('counters');
	$counters->child_add_string('counter', 'wv_fsinfo_blks_total');
	$request->child_add($counters);
 
	my $instances = NaElement->new('instances');
	$instances->child_add_string('instance', $volume_name);
	$request->child_add($instances);
 
	my $result = $server->invoke_elem($request);
	my $blks_total = $result->child_get('instances')->child_get('instance-data')->child_get('counters')->child_get('counter-data')->child_get_int('value');
 
	return $blks_total * 4096;
 
}
 
#
# Returns true if fixed size volume, false if not
# 
sub get_vol_fixedsize {
	my ($server, $volume_name) = @_;
 
	my $request = NaElement->new('volume-size');
	$request->child_add_string('volume', $volume_name);
 
	my $result = $server->invoke_elem($request);
 
	return ( $result->child_get_string('is-fixed-size-flex-volume') eq "false" ? 0 : 1 );
}
 
#
# Returns true if snapmirror volume, false if not
#
sub get_vol_snapmirror {
	my ($server, $volume_name) = @_;
 
	my $request = NaElement->new('snapmirror-get-volume-status');
	$request->child_add_string('volume', $volume_name);
 
	my $result = $server->invoke_elem($request);
 
	# if the filer is not licensed for snapmirror, this request will fail
	if ($result->results_status() eq "failed" && $result->results_reason() =~ /requires license/) {
		return 0;
	} else {
		return ( $result->child_get_string('is-destination') eq "false" ? 0 : 1 );
	}
 
}
 
#
# Defines and parses the options from the command line, also
# checks to make sure that options are valid.  Will prompt for
# a password if one is not provided.
#
sub parse_options {
 
	my %options = (
		'hostname' => '',
		'username' => '',
		'password' => '',
		'help'     => 0,
		'verbose'  => 0
	);
 
	GetOptions( \%options,
		'hostname|H=s',
		'username|u=s',
		'password|p:s',
		'help|h',
		'verbose+'
	);
 
	if (! $options{ 'hostname' } || ! $options{ 'username' } || $options{ 'help' }) {
		print_usage();
		exit(1);
	}
 
	if (! $options{ 'password' }) {
		print "Enter password: ";
		if ( $^O eq "MSWin32" ) {
			require Term::ReadKey;
 
			Term::ReadKey->import( qw(ReadMode) );
			Term::ReadKey->import( qw(ReadLine) );
			ReadMode('noecho');
 
			chomp( $options{ 'password' } = ReadLine(0) );
 
			ReadMode('normal');
 
		} else {
			system("stty -echo") and die "ERROR: stty failed\n";
 
			chomp ( $options{ 'password' } = <STDIN> );
 
			system("stty echo") and die "ERROR: stty failed\n";
 
		}
 
		print "\n";
	}
 
	return \%options;
}
 
sub print_usage {
	print <<EOU
  Missing or incorrect arguments!
 
  na-aggrSpaceCheck.pl --hostname|-H <hostname> 
            --username|-u <username> 
            --password|-p <password> 
 
EOU
 
}
 
#
# Creates the NaServer object
#
sub get_filer {
	my ($hostname, $username, $password) = @_;
 
	my $s = NaServer->new($hostname, 1, 7);
	$s->set_style(LOGIN_PASSWORD);
	$s->set_admin_user($username, $password);
	$s->set_transport_type(NA_SERVER_TRANSPORT_HTTP);
 
	return $s;
}
 
#
# Taken from the source script, this prints the size of the volume
# is a human readable format
#
sub get_printable_size {
	my $size = shift;
	my $size_string;
 
    # Bytes
    if ($size / 1024 < 1 ) {
		$size_string = sprintf("%d bytes", $size);
    }
 
    # Kilo bytes
    elsif ($size / (1024**2) < 1 ) {
		$size_string = sprintf("%4.2fKB", $size / 1024);
    }
 
    # Mega bytes
    elsif ($size / (1024**3) < 1 ) {
		$size_string = sprintf("%4.2fMB", $size / (1024**2));
    }
 
    # Giga bytes
    elsif ($size / (1024**4) < 1 ) {
		$size_string = sprintf("%4.2fGB", $size / (1024**3));
    }
 
    # Tera bytes
    elsif ($size / (1024**5) < 1 ) {
		$size_string = sprintf("%4.2fTB", $size / (1024**4));
    }
 
    # Peta bytes
    elsif ($size / (1024**6) < 1 ) {
		$size_string = sprintf("%4.2fPB", $size / (1024**5));
    }
 
    else {
		$size_string = sprintf("%d bytes", $size);
    }
 
    return $size_string;
}