Showing posts with label edge services gateway. Show all posts
Showing posts with label edge services gateway. Show all posts

Thursday, June 14, 2018

Automating NSX with PowerShell and RESTful API

Often, there are situations when one needs to do the same actions over and over again. System admins solved this repetitive tasks by scripting them. Scripting languages were however limited to the operating system or in some cases to a middleware application. New applications provide RESTful APIs that extend the power of scripting languages from the OS level to virtually every application available in the infrastructure. Using RESTful APIs one can create complex workflows that execute orchestrated actions on different applications. This brings a new word into the vocabulary: automation.

Let's take a simple example - new developer is hired and needs a dedicated dev and test environment. We are using VMware vSphere and NSX to provide that environment. Because we don't want to interfere with other environments, we want to isolate it. For isolating environments at network level we can use Edge Services Gateway (ESG) and dedicated VXLAN. At vSphere level we need to control resource consumption. If there is vRealize Automation or vCloud Director, these tasks can be easily automated from the GUI. What is there is no Cloud Management Portal, or the one the exists is not integrated with NSX and vSphere. Then we need to go back to scripting language.

In this post we'll take a look at how to call NSX RESTful API using PowerShell. There are languages that can be used: ruby, perl, python, JavaScript, depends on the preference. There is also a PowerShell extension for NSX called PowerNSX. The scope of the post is to see how we can use PowerShell to consume RESTful APIs (as the principles will apply to any API) using NSX.


Before we begin, a few words about the environment. I am using PowerShell 6.0.1, downloaded latest stable version from GitHub. The reason for doing this is that in version prior to 6.x there are issues with getting PowerShell methods to trust self signed SSL certificates. In 6.x you can add  -SkipCertificateCheck parameter to both Invoke-RestMethod and Invoke-WebRequest to accept self signed certificates.



Any RESTful API connection needs to be authenticated. Let's create the authentication header that will be sent with the request. We need to get the username and password and encode it to Base 64 string. The password can be simply put as a string in code, but I would like to complicate things and request the password from the user as a secure string (characters masked with star symbols)

1
2
$username = "nsxAdmin"
$securedValue = Read-Host "Enter password" -AsSecureString

After we get the password, we need to decrypt the secure string from unmanaged memory and write it in a string variable. We use the following methods: PtrToStringAuto and SecureStringToBSTR.

1
2
$password = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($securedValue))
$userpass  = $username + ":" + $password

We encode the username and password to Base64 and create the header.

1
2
3
4
5
$bytes= [System.Text.Encoding]::UTF8.GetBytes($userpass)
$encodedlogin=[Convert]::ToBase64String($bytes)
$authheader = "Basic " + $encodedlogin
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$authheader)

Lastly we create the URI (will get existing edges) and run the query. The output will be sent to a XML file.


1
2
3
$uri = "https://nsx-manager/api/4.0/edges"
$outXml = $baseFilePath + 'allEsg.xml'
Invoke-RestMethod -Uri $uri -Headers $headers -Method 'GET' -OutFile $outXml -SkipCertificateCheck

We have the XML file with all existing edges. Let's get vnic configuration for each edge. First we load the XML file and look for edge child nodes - pagedEdgeList.edgePage.edgeSummary. For each edge we find, we use the objectId to query the API for its vnics and put that result into a file.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
$esgObjectArray = @()
[xml]$config = Get-Content $outXml
foreach ($esg in $config.pagedEdgeList.edgePage.edgeSummary) {
  $esgObject = New-Object PSObject -property @{Id=$esg.objectId;Name=$esg.name}

  $uri = 'https://nsx-manager/api/4.0/edges/' + $esg.objectId + '/vnics'
  $outXml = $baseFilePath + $esg.objectId + '-vnics.xml'
  try {
    Invoke-RestMethod -Uri $uri -Headers $headers -Method 'GET' -OutFile $outXml -ea stop -SkipCertificateCheck
  }
  catch {
    Write-Host $_ -foreground green
  }

  $esgObjectArray += $esgObject
}

Within the foreach loop we are creating an array ($esgObjectArray) that contains the edge id and the edge name. $esgObject object is used to get the parameters for each iteration of the loop and add them to the array. This will be used to get the IP addresses for each edge and export all in a CSV file.

The variable $baseFilePath can be any destination on your local system where you want to place output files (e.g "D:\scripts\myEsgConfig\").

Finally we'll export edge ID, edge name and its IP addresses to a csv file. We'll loop through $esgObjectArray and get the information from the saved xml configuration files for each edge (-vnics.xml files):


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
$esgVnics = @()
foreach ($esg in $esgObjectArray) {
  $outXml = $baseFilePath + $esg.Id + '-vnics.xml'
  if (Test-Path -Path $outXml) {
    $vnics = ""
    $item = ""
    [xml]$config = Get-Content $outXml
    foreach ($interface in $config.vnics.vnic) {
      $vnics += "," + $interface.addressGroups.addressGroup.primaryAddress
    }
    $item = $esg.Id + "," + $esg.Name + $vnics
    Write-Host $item
  } else { continue}
  $esgVnics += $item
}

$esgVnics | foreach { Add-Content -Path  $csvFile -Value $_ }

In the end we'll get a CSV file with the following format:

As you've seen, we used Invoke-RestMethod to query the API and saved the results in XML files for later use. You can use other REST methods like PUT, POST or DELETE to manage NSX environment. In the end it is all about manipulating creating and manipulating XML/JSON content to do the operations from command line .

Friday, May 4, 2018

NSX integration with vRealize Automation 7.4 - part 1

From time to time I find myself facing a new configuration of vRealize Automation. And since I don't do it very often I also find myself having forgotten some steps. For this reason I will write down my typical integration of NSX with vRA.

The post has two parts:

  • part 1 - describes the vRA configuration at infrastructure/tenant level
  • part 2 - focuses on creation of the service that consumes NSX


Requirements:

  • NSX  ( > 6.3.x) is installed and configured (VXLAN, distributed logical router, edge services gateways)
  • vRA 7.4 is deployed, tenant created, user directory integrated (optional) 
  • familiarity with vRA 


Goal: 

  • all vRA workloads are deployed to on-demand created networks and we do not worry about routing or virtual network creation
  • security for workloads is ensured using distributed firewall and security tags (but more on this in another post). 


First thing first, drawing a small diagram (sometimes my creative side kicks in :-)) of the desired state:


To solve L2/L3 requirements, we need the following:

  • vRA uses on demand created VXLAN's that use Distributed Logical Router (DLR) as default gateway
  • DLR is connected to Edge Services Gateway (ESG) via a transit network
  • dynamic routing protocol is running between DLR and ESG
  • ESG connects VRA workloads to the rest of the world via the External portgroup (which is a distributed portgroup)
  • ESG may also run dynamic routing protocols
(please don't judge the usage of /24 in the diagram, in real life I am subnetting)

As stated earlier, NSX is already configured, DLR and ESG deployed. Let's see how to configure vRA. 

Logon to the tenant as IaaS Administrator. Ideally, you would have IaaS Administrator + Tenant Administrator roles assigned to your account so you don't need to change between roles. We  need to create vSphere endpoint, fabric group, business group, NSX endpoint, reservation. 



Create vSphere (vCenter) Endpoint:

  • go to Infrastructure > Endpoints > Endpoints
  • New > Virtual > vSphere (vCenter)
  • on General tab: give it a name (vcenter1), add the URL of vCenter Server API (https://vcenter1.mydomain.local/sdk, add credentials and Test Connection. 
  • If all good, press OK and you have created your endpoint.  

Create NSX Endpoint:

  • go to Infrastructure > Endpoints > Endpoints
  • New > Network and Security > NSX
  • on General tab: type the name of the endpoint, add the  URL to the NSX manager and the credentials
  • on Associations tab: map NSX endpoint with the previously created vSphere endpoint (this is a step that appeared in vRA 7.3 due to changes on how NSX is integrated) 

  • press Test Connection, and then OK


Create Fabric Group:
  • go to Infrastructure > Endpoints > Fabric Group > New
  • give it a name, add the Fabric administrators (users, user groups) and select the compute resources available (the list of compute resources is based on the permissions the user has in vCenter Server)


Now the compute resources are available (Infrastructure > Compute Resources > Compute Resources). Check on the compute resource that data collection has run successfully - hover on the compute resource and from the menu choose Data Collection. 

Create Business Group:

  • go to Infrastructure > Users & Groups > Business Groups > New
  • on General tab: type in the name, add an e-mail for alerts, test
  • on Members tab: add the users/user groups for the following roles - Group manager role, Support role, Shared access role,User role and press ok
Starting with vRA 7.3 there is a new role for Business group - Shared access role which can use and run actions on resources deployed by other users in the business group. It is a good addition since I remember a client wanting this back in 2015.  

Create Network Profiles
  • go to Infrastructure > Reservations > Network Profiles
  • New > External 
  • on General tab: for Transit VXLAN: add the name, subnet mask and gateway IP address 
  • on DNS tab: add DNS details
  • on Network Ranges tab: add the IP range that is usable (do not forget that there are 2 IPs already used by ESG and DLR) 
  • New > Routed 
  • on General tab: use the external network profile created previously, and add some subnetting details:
    • subnet mask (could be /24 - 254 IPs) - the whole range given to vRA for its workloads
    • range subnet mask (can be /29 - 6 IPs ) - for each application/group of application deployed
  • on Network Ranges tab: press Generate Ranges
At least the following two network profiles will be displayed:




Create Reservation:
  • go to Infrastructure > Reservations 
  • New >  vSphere (vCenter)
  • on General tab: type in the Name of the reservation, tenant name (if multiple tenants exist), business group for which the reservation is created, priority (in case multiple reservations exist for the same business group)
  • on Resources tab: select the compute resource, put in a machine quota (if needed), select the size of the memory, select the datastores and the storage quota and select the Resource pool (if one has been defined in vCenter Server)
  • on Network tab: select network adapter (transit VXLAN), transport zone, DLR and network profile
  • finalize the task by pressing OK
And we are set: we have compute, storage and network resource available for consumption. In the next post we will create a service and see how we consume NSX on demand.