VMware documentation defines the proper way of restarting components in a vRA distribute environment. What I've done is to actually take those steps and put them in a PowerCLI script making the procedure reusable and predictable. A particular case is to detect if a VM is a placeholder VM (being a VM replica). Before going to the script itself, let's look at the whole workflow.
The first part is just a sequential shutdown and wait until the VMs poweroff to go to the next step. Then a cold snapshot is taken for all VMs. Lastly, VMs are powered on in an orchestrated sequence and wait times are implemented to allow for the services to come back up.
Getting to code part - first we define the list of VRA components, in this case proxies, DEM-workers, DEM-orchestrators, IaaS web, IaaS Managers and vRA Appliances.
# vCenter Servers $vCSNames = ("vcssrv1", "vcssrv2", "vcssrv3","vcssrv4") # vRA Components $workers = @("vradem1", "vradem2","vraprx1","vraprx2", "vraprx3", "vraprx4", "vraprx5", "vraprx6") $managerPrimary = @("vramgr1") $managerSecondary = @("vramgr2") $webPrimary = @("vraweb1") $webSecondary = @("vraweb2") $vraPrimary = @("vraapp1") $vraSecondary = @("vraapp2") # Snapshots $snapName = "vra upgrade" $snapDescription = "before 7.6 upgrade" # Log file $log = "coldSnapshotVra.log"
Next we define the 3 functions for shutdown, snapshot and start the VMs. Since in our environment we use SRM, I had to check for placeholder VMs when powering off and snapshotting the VMs. We'll take them one by one. First. shutdown VMs and wait for the VM to stop:
function shutdownVMandWait($vms,$log) { foreach ($vmName in $vms) { try { $vm = Get-VM -Name $vmName -ErrorAction Stop foreach ($o in $vm) { if($o.ExtensionData.Summary.Config.ManagedBy.Type -eq "placeholderVm") { Write-Host "VM: '$($vmName)' is placeholderVm. Skipping." } else { if (($o.PowerState) -eq "PoweredOn") { $v = Shutdown-VMGuest -VM $o -Confirm:$false Write-Host "Shutdown VM: '$($v.VM)' was issued" Add-Content -Path $log -Value "$($v)" } else { Write-Host "VM '$($vmName)' is not powered on!" } } } } catch { Write-Host "VM '$($vmName)' not found!" } } foreach ($vmName in $vms) { try { $vm = Get-VM -Name $vmName -ErrorAction Stop while($vm.PowerState -eq 'PoweredOn') { sleep 5 Write-Host "VM '$($vmName)' is still on..." $vm = Get-VM -Name $vmName } Write-Host "VM '$($vmName)' is off!" } catch { Write-Host "VM '$($vmName)' not found!" } } }
Next, take snapshots of the VMs
function snapshotVM($vms,$snapName,$snapDescription,$log) { foreach ($vmName in $vms) { try { $vm = Get-VM -Name $vmName -ErrorAction Stop } catch { Write-Host "VM '$($vmName)' not found!" Add-Content -Path $log -Value "VM '$($vmName)' not found!" } try { foreach ($o in $vm) { if($o.ExtensionData.Summary.Config.ManagedBy.Type -eq "placeholderVm") { Write-Host "VM: '$($vmName)' is placeholderVm. Skipping." Add-Content -Path $log -Value "VM: '$($vmName)' is placeholderVm. Skipping." } else { New-Snapshot -VM $o -Name $snapName -Description $snapDescription -ErrorAction Stop } } } catch { Write-Host "Could not snapshot '$($vmName)' !" Add-Content -Path $log -Value "Could not snapshot '$($vmName)' !" } } }
And finally, power on the VMs:
function startupVM($vms,$log) { foreach ($vmName in $vms) { try { $vm = Get-VM -Name $vmName -ErrorAction Stop foreach ($o in $vm) { if($o.ExtensionData.Summary.Config.ManagedBy.Type -eq "placeholderVm") { Write-Host "VM: '$($vmName)' is placeholderVm. Skipping." } else { if (($o.PowerState) -eq "PoweredOff") { Start-VM -VM $o -Confirm:$false -RunAsync } else { Write-Host "VM '$($vmName)' is not powered off!" } } } } catch { Write-Host "VM '$($vmName)' not found!" } } }
Last part of the script is the putting all the logic together. Connect to vCenter Server, orderly shutdown VMs, take the cold snapshots and bringing back the whole environment.
# MAIN # Connect vCenter Server $creds = Get-Credential try { Connect-VIServer $vCSNames -Credential $creds } catch { Write-Host $_.Exception.Message } # Stop VRA VMs Write-Host "### Stopping DEM Workers an Proxies" shutdownVMandWait -vms $workers -log $log Write-Host "### Stopping Secondary Managers and Orchestrators" shutdownVMandWait -vms $managerSecondary -log $log Write-Host "### Stopping Primary Managers and Orchestrators" shutdownVMandWait -vms $managerPrimary -log $log Write-Host "### Stopping secondary Web" shutdownVMandWait -vms $webSecondary -log $log Write-Host "### Stopping primary Web" shutdownVMandWait -vms $webPrimary -log $log Write-Host "### Stopping secondary VRA" shutdownVMandWait -vms $vraSecondary -log $log Write-Host "### Stopping primary VRA" shutdownVMandWait -vms $vraPrimary -log $log # Snapshot VRA VMs Write-Host "### Snapshotting DEM Workers an Proxies" snapshotVM -vms $workers -snapName $snapName -snapDescription $snapDescription -log $log Write-Host "### Snapshotting Secondary Managers and Orchestrators" snapshotVM -vms $managerSecondary -snapName $snapName -snapDescription $snapDescription -log $log Write-Host "### Snapshotting Primary Managers and Orchestrators" snapshotVM -vms $managerPrimary -snapName $snapName -snapDescription $snapDescription -log $log Write-Host "### Snapshotting secondary Web" snapshotVM -vms $webSecondary -snapName $snapName -snapDescription $snapDescription -log $log Write-Host "### Snapshotting primary Web" snapshotVM -vms $webPrimary -snapName $snapName -snapDescription $snapDescription -log $log Write-Host "### Snapshotting secondary VRA" snapshotVM -vms $vraSecondary -snapName $snapName -snapDescription $snapDescription -log $log Write-Host "### Snapshotting primary VRA" snapshotVM -vms $vraPrimary -snapName $snapName -snapDescription $snapDescription -log $log # Start VRA VMs Write-Host "### Starting primary VRA" startupVM -vms $vraPrimary -log $log Write-Host " Sleeping 5 minutes until Licensing service is registered" Start-Sleep -s 300 Write-Host "### Starting secondary VRA" startupVM -vms $vraSecondary -log $log Write-Host " Sleeping 15 minutes until ALL services are registered" Start-Sleep -s 900 Write-Host "### Starting Web" startupVM -vms $webPrimary -log $log startupVM -vms $webSecondary -log $log Write-Host " Sleeping 5 minutes until services are up" Start-Sleep -s 300 Write-Host "### Starting Primary manager" startupVM -vms $managerPrimary -log $log Write-Host " Sleeping 3 minutes until manager is up" Start-Sleep -s 180 Write-Host "### Starting Secondary manager" startupVM -vms $managerSecondary -log $log Write-Host " Sleeping 3 minutes until manager is up" Start-Sleep -s 180 Write-Host "### Starting DEM Workers an Proxies" startupVM -vms $workers -log $log Write-Host "### All components have been started" # Disconnect vCenter Disconnect-VIServer * -Confirm:$false
You will notice that the orchestration logic is actually implemented here. This means you can easily add/remove/modify the VMs that the script targets. Let's say you only want to snapshot some proxies for which you don't need to bring everything down. Or you want to add external vRealize Orchestrators appliances. All changes take place in the main part by simply commenting out some steps.
This script helped a lot the nightly operations we had to do across our whole environment and I hope it will do the same for you.
No comments:
Post a Comment