Saturday, June 30, 2018

VMware BIOS UUID, vCloud Director and Veeam Agent

Looking at the title it may seem a long and complicated story. Actually is a very simple one. While working in lab which is hosted by a vCloud Director (vCD) instance, I was testing a SQL Always ON Cluster and the Veeam Agent for Windows. I've created my 3 node cluster, installed the database, configured availability groups.  All went well until I tried to install the Veeam agent. This stopped with the error that  the same UUID was being utilized by all 3 nodes.

Well, all 3 VMs were deployed from the same vCD catalog VM. According to this VMware KB article, by default vCD keeps the same UUID value for cloned VM's. Since I did not have access to vCD to change the settings, the only choice was to actually modify the UUID of the VMs.

There are several ways of doing in it, from manual to programmatic (as can be seen in this KB article)

I've chosen PowerCLI variant. The steps are pretty straight forward:
  • shutdown the VM
  • get the current UUID
  • change the UUID
  • power on the VM

And the code to do it below. Do not forget to change the $newUuid (I used to modify the last digits from the current one).


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
$vmName = "myClonedVm"
$vm = Get-VM -Name $vmName

$vm.extensiondata.config.uuid
$newUuid = "00112233-4455-6677-8899-aabbccddeeff"

$vm | Shutdown-VMGuest
While ((Get-VM -Name $vmName).PowerState -ne "PoweredOff") {
  Write-Host -foreground yellow "... waiting for" $vmName "to power off"
  sleep 5
}

$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$spec.uuid = $newUuid
$vm.Extensiondata.ReconfigVM_Task($spec)

Start-VM -VM $vmName -RunAsync

Thursday, June 14, 2018

Automating NSX with PowerShell and RESTful API

Often, there are situations when one needs to do the same actions over and over again. System admins solved this repetitive tasks by scripting them. Scripting languages were however limited to the operating system or in some cases to a middleware application. New applications provide RESTful APIs that extend the power of scripting languages from the OS level to virtually every application available in the infrastructure. Using RESTful APIs one can create complex workflows that execute orchestrated actions on different applications. This brings a new word into the vocabulary: automation.

Let's take a simple example - new developer is hired and needs a dedicated dev and test environment. We are using VMware vSphere and NSX to provide that environment. Because we don't want to interfere with other environments, we want to isolate it. For isolating environments at network level we can use Edge Services Gateway (ESG) and dedicated VXLAN. At vSphere level we need to control resource consumption. If there is vRealize Automation or vCloud Director, these tasks can be easily automated from the GUI. What is there is no Cloud Management Portal, or the one the exists is not integrated with NSX and vSphere. Then we need to go back to scripting language.

In this post we'll take a look at how to call NSX RESTful API using PowerShell. There are languages that can be used: ruby, perl, python, JavaScript, depends on the preference. There is also a PowerShell extension for NSX called PowerNSX. The scope of the post is to see how we can use PowerShell to consume RESTful APIs (as the principles will apply to any API) using NSX.


Before we begin, a few words about the environment. I am using PowerShell 6.0.1, downloaded latest stable version from GitHub. The reason for doing this is that in version prior to 6.x there are issues with getting PowerShell methods to trust self signed SSL certificates. In 6.x you can add  -SkipCertificateCheck parameter to both Invoke-RestMethod and Invoke-WebRequest to accept self signed certificates.



Any RESTful API connection needs to be authenticated. Let's create the authentication header that will be sent with the request. We need to get the username and password and encode it to Base 64 string. The password can be simply put as a string in code, but I would like to complicate things and request the password from the user as a secure string (characters masked with star symbols)

1
2
$username = "nsxAdmin"
$securedValue = Read-Host "Enter password" -AsSecureString

After we get the password, we need to decrypt the secure string from unmanaged memory and write it in a string variable. We use the following methods: PtrToStringAuto and SecureStringToBSTR.

1
2
$password = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($securedValue))
$userpass  = $username + ":" + $password

We encode the username and password to Base64 and create the header.

1
2
3
4
5
$bytes= [System.Text.Encoding]::UTF8.GetBytes($userpass)
$encodedlogin=[Convert]::ToBase64String($bytes)
$authheader = "Basic " + $encodedlogin
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$authheader)

Lastly we create the URI (will get existing edges) and run the query. The output will be sent to a XML file.


1
2
3
$uri = "https://nsx-manager/api/4.0/edges"
$outXml = $baseFilePath + 'allEsg.xml'
Invoke-RestMethod -Uri $uri -Headers $headers -Method 'GET' -OutFile $outXml -SkipCertificateCheck

We have the XML file with all existing edges. Let's get vnic configuration for each edge. First we load the XML file and look for edge child nodes - pagedEdgeList.edgePage.edgeSummary. For each edge we find, we use the objectId to query the API for its vnics and put that result into a file.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
$esgObjectArray = @()
[xml]$config = Get-Content $outXml
foreach ($esg in $config.pagedEdgeList.edgePage.edgeSummary) {
  $esgObject = New-Object PSObject -property @{Id=$esg.objectId;Name=$esg.name}

  $uri = 'https://nsx.eudemo.veeam.local/api/4.0/edges/' + $esg.objectId + '/vnics'
  $outXml = $baseFilePath + $esg.objectId + '-vnics.xml'
  try {
    Invoke-RestMethod -Uri $uri -Headers $headers -Method 'GET' -OutFile $outXml -ea stop -SkipCertificateCheck
  }
  catch {
    Write-Host $_ -foreground green
  }

  $esgObjectArray += $esgObject
}

Within the foreach loop we are creating an array ($esgObjectArray) that contains the edge id and the edge name. $esgObject object is used to get the parameters for each iteration of the loop and add them to the array. This will be used to get the IP addresses for each edge and export all in a CSV file.

The variable $baseFilePath can be any destination on your local system where you want to place output files (e.g "D:\scripts\myEsgConfig\").

Finally we'll export edge ID, edge name and its IP addresses to a csv file. We'll loop through $esgObjectArray and get the information from the saved xml configuration files for each edge (-vnics.xml files):


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
$esgVnics = @()
foreach ($esg in $esgObjectArray) {
  $outXml = $baseFilePath + $esg.Id + '-vnics.xml'
  if (Test-Path -Path $outXml) {
    $vnics = ""
    $item = ""
    [xml]$config = Get-Content $outXml
    foreach ($interface in $config.vnics.vnic) {
      $vnics += "," + $interface.addressGroups.addressGroup.primaryAddress
    }
    $item = $esg.Id + "," + $esg.Name + $vnics
    Write-Host $item
  } else { continue}
  $esgVnics += $item
}

$esgVnics | foreach { Add-Content -Path  $csvFile -Value $_ }

In the end we'll get a CSV file with the following format:

As you've seen, we used Invoke-RestMethod to query the API and saved the results in XML files for later use. You can use other REST methods like PUT, POST or DELETE to manage NSX environment. In the end it is all about manipulating creating and manipulating XML/JSON content to do the operations from command line .

Friday, June 1, 2018

PowerCLI - Get the sizing of virtual machines

Any backup project needs an answer to at least the following questions: what is the total number of VMs, what is total backup size, what is the number of virtual disks per VM. It doesn't mean that it is that easy, since there are more questions that need answers, but these three are the base. Sometimes there is a fast answer to them, but there are situations when the information is unknown, or more difficult to find. To make life easier, I've put together a small script that is based on PowerCLI commandlets Get-VM and Get-HardDisk.

The script runs against vCenter Server, retrieves each VM, parses the data and sends the needed information to a CSV file. The file has the following structure: VM name, number of disks attached to the VM, used space without swap file (in GB), total used space (in GB), provisioned space (in GB).

A short explanation on the 3 different values for space:

  • provisioned space is the space requested by the VM on the datastore. For thick provisioned VMs the allocated (used) space is equal to requested space. Thin provisioned ones do not receive the space unless they consume it. Hence the difference between provisioned space and used space (allocated) columns
  • disk space reported in used column contains other files than the VM disks (vmdk). One of this files is the VM swap file, which is equal to the difference between the VM memory and the amount of VM reserved memory. This means that if there are no reservations on the VM memory, each VM will have the swap space equal to the size of memory. This file is not part of the backup and can be excluded. If we do a math exercise - 150 VMs with 8 GB each means more than 1 TB of data which we can ignore from calculations
  • finally, looking at vm10 in the image above, we see the 2 columns (used and used without swap) being equal. Since vm10 is powered on, this means it has a memory reservation equal to the whole memory. For powered off VMs, the 2 columns will always be equal since swap file is created only for running VMs.
Let's take a look at the code. First we define a function that takes as input a VM object and returns a custom made power shell object with the required properties of the VM

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
function GetVMData($v) {
 $vmResMemory = [math]::Round($v.ExtensionData.ResourceConfig.MemoryAllocation.Reservation/1024,2)
 $vmMem = [math]::Round($v.MemoryMB/1024,2)
 $vmUsedSpace = [math]::Round($v.UsedSpaceGB,2)
 if ($v.PowerState -match "PoweredOn") {
  $vmUsedSpaceNoSwap = $vmUsedSpace - $vmMem + $vmResMemory # removing swap space from calculations
 } else {
  $vmUsedSpaceNoSwap = $vmUsedSpace
 }
 $vmProvSpace = [math]::Round($v.ProvisionedSpaceGB,2) # swap space included
 $vmName = $v.Name
 $vmNoDisks = ($v | Get-HardDisk).count

 $hash = New-Object PSObject -property @{Vm=$v.Name;NoDisks=$vmNoDisks;UsedSpaceNoSwap=$vmUsedSpaceNoSwap;UsedSpace=$vmUsedSpace;ProvSpace=$vmProvSpace}
 return $hash

}

We look at several parameters of the VM: memory reservation, allocated memory, used space, number of disks. We take the parameters and create a new PowerShell object that the function returns as a result.

The main body of the script is pretty simple. First, we define the format of the output CSV file and the we take every VM from vCenter Server (you need to be connected to vCenter Server before running this script) and process it:


1
2
3
4
5
6
7
8
9
$vmData = @('"Name","NoDisks","UsedSpaceGB(noSwap)","UsedSpaceGB","ProvisionedSpaceGB"')
$csvFile = ($MyInvocation.MyCommand.Path | Split-Path -Parent)+"\vmData.csv"

foreach ($v in get-vm) {
        $hash = GetVMData -v $v
 $item = $hash.Vm + "," + $hash.NoDisks + "," + $hash.UsedSpaceNoSwap + "," + $hash.UsedSpace + "," + $hash.ProvSpace
 $vmData += $item
}
$vmData | foreach { Add-Content -Path  $csvFile -Value $_ }

If you want to look only at VMs that are powered on, then you need to add an IF clause that checks the VM power state in the FOR loop before processing each VM:


1
2
3
if ($v.PowerState -match "PoweredOn") {
# process $v
}

Not it's time to look at the output CSV file and start sizing for that backup solution.