Showing posts with label REST. Show all posts
Showing posts with label REST. Show all posts

Wednesday, December 2, 2020

Using NSX load balancer as a monitoring tool for RESTful APIs

We are going to look at a different use case for NSX load balancer - monitoring tool for external API's. 

Our core platform is integrating with other systems using RESTful API's. These systems even though they are built with high availability in mind, they are sometimes highly unavailable. They are also part of the critical path for our core platform. Not being able to reach the systems creates troubles  in the form of incident tickets because we fail to deliver services to our customers. So we needed a way to monitor those API's.

We know that the systems are monitored, but we don't have access to those tools. We have our own tools, but they do not offer a simple and efficient way to check the status of a RESTful API. Ideally we don't want to introduce another monitoring tool. However, the core platform is running on top of NSX and it actually uses NSX load balancers for its internal service. So why not use load balancers to monitor the external services? 

We created a service monitor and a pool in the load balancer for each of the external systems. This way NSX monitors the status of the RESTful API of the system and generates alerts whenever it is down. The status of the pool is then checked by the core platform. All communication between core platform and APIs goes directly. It does not use the load balancer.

The pool contains the RESTful API endpoint of the system that we use to connect directly from the core platform. 

The service monitor uses GET requests to check the availability of the RESTful API. 


Nothing fancy, basic configuration for a load balancer. Half-configuration actually because here we stop as no traffic goes through the load balancer to these pools. But whenever the external system is not reachable, the load balancer knows it because now the external system is a member in on of its pools: 


The status of the member in the pool is accessible through the RESTful API of NSX Manager. 

GET /api/4.0/edges/{edge-id}/loadbalancer/statistics

<status>DOWN</status>
<failureCause>layer 7 response error, code:400 Bad Request</failureCause>
<lastStateChangeTime>2020-12-02 18:20:43</lastStateChangeTime>

This way the core platform knows the status of its external systems before doing anything. More important core platform can now act on that status. In this case it will wait a specific period of time until trying again to use the system. 

It is a pretty simple solution. It is also pretty obvious that the APIs should have been monitored. We actually relied too much on the availability of those API's and used a fire and forget approach. The approach was far from optimal and it impacted our KPIs and created additional operational workload. 

Thursday, June 14, 2018

Automating NSX with PowerShell and RESTful API

Often, there are situations when one needs to do the same actions over and over again. System admins solved this repetitive tasks by scripting them. Scripting languages were however limited to the operating system or in some cases to a middleware application. New applications provide RESTful APIs that extend the power of scripting languages from the OS level to virtually every application available in the infrastructure. Using RESTful APIs one can create complex workflows that execute orchestrated actions on different applications. This brings a new word into the vocabulary: automation.

Let's take a simple example - new developer is hired and needs a dedicated dev and test environment. We are using VMware vSphere and NSX to provide that environment. Because we don't want to interfere with other environments, we want to isolate it. For isolating environments at network level we can use Edge Services Gateway (ESG) and dedicated VXLAN. At vSphere level we need to control resource consumption. If there is vRealize Automation or vCloud Director, these tasks can be easily automated from the GUI. What is there is no Cloud Management Portal, or the one the exists is not integrated with NSX and vSphere. Then we need to go back to scripting language.

In this post we'll take a look at how to call NSX RESTful API using PowerShell. There are languages that can be used: ruby, perl, python, JavaScript, depends on the preference. There is also a PowerShell extension for NSX called PowerNSX. The scope of the post is to see how we can use PowerShell to consume RESTful APIs (as the principles will apply to any API) using NSX.


Before we begin, a few words about the environment. I am using PowerShell 6.0.1, downloaded latest stable version from GitHub. The reason for doing this is that in version prior to 6.x there are issues with getting PowerShell methods to trust self signed SSL certificates. In 6.x you can add  -SkipCertificateCheck parameter to both Invoke-RestMethod and Invoke-WebRequest to accept self signed certificates.



Any RESTful API connection needs to be authenticated. Let's create the authentication header that will be sent with the request. We need to get the username and password and encode it to Base 64 string. The password can be simply put as a string in code, but I would like to complicate things and request the password from the user as a secure string (characters masked with star symbols)

1
2
$username = "nsxAdmin"
$securedValue = Read-Host "Enter password" -AsSecureString

After we get the password, we need to decrypt the secure string from unmanaged memory and write it in a string variable. We use the following methods: PtrToStringAuto and SecureStringToBSTR.

1
2
$password = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($securedValue))
$userpass  = $username + ":" + $password

We encode the username and password to Base64 and create the header.

1
2
3
4
5
$bytes= [System.Text.Encoding]::UTF8.GetBytes($userpass)
$encodedlogin=[Convert]::ToBase64String($bytes)
$authheader = "Basic " + $encodedlogin
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$authheader)

Lastly we create the URI (will get existing edges) and run the query. The output will be sent to a XML file.


1
2
3
$uri = "https://nsx-manager/api/4.0/edges"
$outXml = $baseFilePath + 'allEsg.xml'
Invoke-RestMethod -Uri $uri -Headers $headers -Method 'GET' -OutFile $outXml -SkipCertificateCheck

We have the XML file with all existing edges. Let's get vnic configuration for each edge. First we load the XML file and look for edge child nodes - pagedEdgeList.edgePage.edgeSummary. For each edge we find, we use the objectId to query the API for its vnics and put that result into a file.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
$esgObjectArray = @()
[xml]$config = Get-Content $outXml
foreach ($esg in $config.pagedEdgeList.edgePage.edgeSummary) {
  $esgObject = New-Object PSObject -property @{Id=$esg.objectId;Name=$esg.name}

  $uri = 'https://nsx-manager/api/4.0/edges/' + $esg.objectId + '/vnics'
  $outXml = $baseFilePath + $esg.objectId + '-vnics.xml'
  try {
    Invoke-RestMethod -Uri $uri -Headers $headers -Method 'GET' -OutFile $outXml -ea stop -SkipCertificateCheck
  }
  catch {
    Write-Host $_ -foreground green
  }

  $esgObjectArray += $esgObject
}

Within the foreach loop we are creating an array ($esgObjectArray) that contains the edge id and the edge name. $esgObject object is used to get the parameters for each iteration of the loop and add them to the array. This will be used to get the IP addresses for each edge and export all in a CSV file.

The variable $baseFilePath can be any destination on your local system where you want to place output files (e.g "D:\scripts\myEsgConfig\").

Finally we'll export edge ID, edge name and its IP addresses to a csv file. We'll loop through $esgObjectArray and get the information from the saved xml configuration files for each edge (-vnics.xml files):


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
$esgVnics = @()
foreach ($esg in $esgObjectArray) {
  $outXml = $baseFilePath + $esg.Id + '-vnics.xml'
  if (Test-Path -Path $outXml) {
    $vnics = ""
    $item = ""
    [xml]$config = Get-Content $outXml
    foreach ($interface in $config.vnics.vnic) {
      $vnics += "," + $interface.addressGroups.addressGroup.primaryAddress
    }
    $item = $esg.Id + "," + $esg.Name + $vnics
    Write-Host $item
  } else { continue}
  $esgVnics += $item
}

$esgVnics | foreach { Add-Content -Path  $csvFile -Value $_ }

In the end we'll get a CSV file with the following format:

As you've seen, we used Invoke-RestMethod to query the API and saved the results in XML files for later use. You can use other REST methods like PUT, POST or DELETE to manage NSX environment. In the end it is all about manipulating creating and manipulating XML/JSON content to do the operations from command line .