Monday, December 31, 2018

Role Based Access for VMware in Veeam Backup & Replication 9.5 Update 4

One of the cool features that Veeam Backup & Replication 9.5 Update 4 comes with is integration with vCenter Server role based access. What does it mean? It allows to delegate permissions to users and groups in Veeam Enterprise Manager based on their permissions in vCenter Server.

A user or group of users is now able to monitor and control the backup and restore of their own VMs in a vSphere environment based on a predefined policy. A policy can be defined through vSphere tags, a role in vCenter or as granular as a single permission in vCenter Server. Delegation is done through the self service portal in  Enterprise Manager.

Cool thing is the integration is actually extending vCenter Server access control by adding vSphere tags as control mechanisms. For example DBA's want to do their backup and restore, then just assign a tag to the VMs and create the policy in Enterprise Manager. It's that simple.

Since my environment uses tags, we will test the following scenario: all developers will have access to development VMs which are tagged with "SLA3" vSphere tags.

First, make sure tags exist in vCenter Server and are assigned to the VMs in scope.

Next, you need to install and configure VBR and Enterprise Manager (EM). This is not in the scope of the current article.

Once EM is installed and configured  login to the EM portal (https://em-address:9443/) and go to Configuration. Check VBR and vCenter Server are available and reachable.

On Self Service tab you will see the default configuration for all Domain Users group (my lab is AD integrated).

For the test we will create a new configuration.

Let's look at the Delegation Mode - what mechanism we use to define access:

By default, it uses VM privilege and the selected privilege is VirtualMachine.Interact.Backup, but you can choose to use any privilege available in vCenter Server. Once you need more flexibility, you can define roles in vCenter Server and use the delegation based on those roles (a set of privileges applied to an object). Finally, you can use vSphere tags and allow access based on the tags. Once the preferred method of delegation is chosen, it will apply to all self service configurations. So be careful when changing the method. 

Now open, the default self service configuration and let's take a look at it. We see it assigns a repository and a quota on that repository. The quota can be global (for all users in group) or individual (per each user). 

You also define the default settings for advanced job settings such as compression , deduplication, scheduling of full backups and so on. The settings can be copied from Veeam default or from an existing job. 

There are 4 job scheduling options available from allowing full access to the scheduler to no access to the scheduler. We will use the default one (full access to job scheduling). Choose wisely what you want your users to do.

vSphere tags drop down list appeared because I chose the Delegation Method as vSphere tags, but it's left empty. 

Let's create a new self service configuration for developers group. Press Add and then:
1. select the Type - user or group and search it in AD
2. select the repository and define the quota
3. select job scheduling options
4. select the vSphere tag
5. configure the advanced settings

Press Save and open the self service portal (https://em-address:9443/backup/). Login with one of the users from the group (Developers in my case). Since user is member of 2 configurations, select which configuration to logon:

Once logged in, the portal displays 5 tabs: Dashboards, Jobs, VMs, Files and Items

Go to Jobs and create a new backup job. A process similar to VBR console job creation will start. First give the job a name, description and decide how many restore points to keep:

Then add the VMs you want to backup. Only VMs with SLA3 tag will be displayed.

If required, enable application aware processing and add credentials for guest processing:

Schedule the job (in this configuration you are allowed):

Lastly, enable notifications (if you want to be alerted about the status of the job)

Now that the job has been created, you can run it:

Meanwhile, we can take a look at how things look in VBR console

You'll notice the running job named with the following convention: Domain\GroupName_Domain\UserName_JobName. So, all is good and running smoothly.

Back in the self service portal, pressing on the job statistics we can see what happened:

There you go, just had a user define a backup job and backup its own VM using a simple vSphere tag and no other settings at either vCenter Server or VBR level. Next time we'll take a look at restore options.

Tuesday, December 4, 2018

Configure Veeam Backup for Microsoft Office 365

In the previous post, we've gone through the steps to install VBO365 on a Windows Core VM. Now we'll look at configuration.

After installation, a default backup proxy and default backup repository are created on the server. Proxies are responsible for handling backup traffic from Office 365 to repository and restore traffic from repository to Veeam explorers. For small deployments the same server (management server) can be used as proxy and repo server. However it is recommended to use external proxies and repositories.

One very important aspect of the repository is that it defines the retention policy. This means that all backup jobs pointing to one repository will get the retention defined at that repository level.

More important, in VBO365 retention policy is defined as the number of years/days since the object has been changed. In the case of a 3 year retention policy, only e-mail that have been touched in the past 3 years will be backed up. In case all emails are required for a certain mailbox, keep forever policy can be selected.

Next we will add the O365 organization that we want to backup

Once the organization has been added, we can create backup jobs. Although it is possible to backup everything in one job, it is recommended to create separate backup jobs for Exchange, SharePoint , Archive and OneDrive and point them to different repositories. 

Next we'll create a backup job for emails of only 4 users in the organization. Start the wizard and give the job a name:

Select the users you want to backup:

Select the items to backup for the users (by default All, we'll select only e-mails):

Add exclusions (in our case not required):

Choose the proxy and repository (remember the repo sets the retention):\

Confirm the job execution schedule:

The job will be created and stopped. Right click on it and start to backup. 

Once the backup finishes, you can right click on the job and open Exchange explorer

Tuesday, November 27, 2018

Enable RDP in SLES 12

Haven't been working with SuSE Linux for a while and when this new project came along I got my hands on SLES 12 SP3.

I've created an account on, downloaded the ISO, created the VM and installed from scratch. It went pretty smoothly. The suse account gives a 60 day trial activation key which will allow to update the server. After the 60 days, no more update, but it's just for a lab setup.

The one thing I need from the server is to provide terminal services. So I installed xrdp from using Software Manager in Yast 2

Next, enable xrdp service and start it:
sudo systemctl enable xrdp
sudo systemctl start xrdp

Last thing to do is to configure the firewall. It is enabled by default and will not allow connection to the server. Check that the xrdp service definition exists. If not create a file that looks like this:

Configure the firewall to accept xrdp (and if you are at it, add ssh also). Open the file /etc/sysconfig/SuSEfirewall2 and change the following lines to include your services:

Save the changes and restart the firewall service:
sudo systemctl restart SuSEfirewall2

Now, if all went good open your preferred RPD client and give it a try:

Friday, September 21, 2018

VCP 6.5 Delta Exam experience

The dreadful reminder of imminent VCP expiration came again. I had already decided to take the VCP delta exam called 2V0-622, without actually doing it. But this time I was prepared. Having happily paid exam fee (using my VMUG Advantage discount), I was waiting for the day to come. It came and passed pretty fast and I did pass the exam.

Must admit I was a bit scared, but it is a delta exam and if you are using the technology on a daily basis then you can't have a lot of surprises. It didn't seem to be a difficult exam to take. Maybe having delivered the ICM 6.5 a few times in my life, had helped. However, I would like to point out several aspects about how I prepared for the exam:

  • never forget configuration maximums - I failed one question because I did not read them properly
  • it is a delta exam - so what's new is important especially since 6.5 was more like a major release bringing in VMFS6, security features, NFS 4.1 and other goodies. If you don't use the features in your production environment, stand up a nested ESXi and play with them a bit (of course, always read the manual)
  • features you don't use in production on day to day basis (let's say VSAN), read the manual
  • use a practice mock exam to help you find your weaknesses and study more (this one from Simon Long is pretty cool)
  • lastly, if you don't know an answer, start and use your logic and what you know to eliminate the wrong answers. This method may lead to finding the good answer
For all of you thinking about  taking the exam, good luck! 

Saturday, September 15, 2018

Secure Boot and Acceptance Levels of Hosts and VIBs

Acceptance levels of VIB  provide an information on the amount of certification the software package has undergone. There are four levels of acceptance for VIBs:

  • VMwareCertified - most stringent requirements equivalent to VMware in-house Quality Assurance testing. Only I/O Vendor Program (IOVP) program drivers are published at this level.VMware takes support calls for VIBs with this acceptance level
  • VMwareAccepted - partner runs the tests and VMware verifies the result. VMware takes support calls for VIBs with this acceptance level.
  • PartnerSupported - The partner performs all testing. VMware does not verify the results. VMware directs support calls for VIBs with this acceptance level to the partner's support organization.
  • CommunitySupported - is for VIBs created by individuals or companies outside of VMware partner programs. They are not supported by VMware or partners.

Why is this interesting for us? Mostly because of the relationship between secure boot and the acceptance level. Secure boot does not allow to set the acceptance level to CommunitySuported. This makes perfect sens, why would you want to install a VIB that is created by someone outside the trusted partner program. Two answers come to mind: home labs and testing.

With secure boot enabled (which is default for VMs created with UEFI in vSphere 6.5 U1) you will notice the following behavior when trying to set the acceptance level:

In order to be able to set the desired acceptance level, you should disable secure boot. If it's a physical server, then you need to do this in UEFI. For VMs, it can be done at VM level, but still needs a power off and then select the VM and edit settings. On VM Options tab, under Boot Options you will find the setting for Secure Boot.

If you are connecting to vSphere 6.5, use web client since HTML one does not show the option.

Wednesday, September 5, 2018

Installing Veeam Backup for Microsoft Office 365 on Windows Server 2016 Core

Windows Server 2016 Core is a minimal installation that has a smaller footprint which translates to a smaller attack surface. Having security in mind, it make perfect sense to use Core for deploying server components on it.

Veeam Backup for Microsoft Office 365 (VBO365) allows to back up and recover Microsoft Office 365 and on-premises Exchange and SharePoint organizations data containing Microsoft Exchange items, Microsoft SharePoint items, and OneDrive documents.

VBO365 is made of several components:
  • VBO365 server 
  • console
  • SharePoint explorer
  • Exchange explorer
  • PowerShell extensions
Since VBO365 is modular and since it needs Internet access, it can be installed on separate machines. In our case, we'll install server and PowerShell extension on a Windows Server 2016 Core situated in DMZ and the console and explorers on the admin workstation in the management area. Repository space will be provisioned on the local disk of VBO365. 

First, we start by deploying a virtual machine with Windows Server 2016 Core installed on it. The VM is deployed from a template created previously. To create the template do the follwing:
  • configure a VM with 1 vCPU, 512 MB of RAM and 32 GB of disk
  • upload the Windows iso file on a datastore
  • attach the VM CDROM to the iso file
  • start the VM and follow the installation steps - basic Windows install, just choose the Windows install without Desktop Experience
  • enter the license key and wait for the install to finish
For a step-by-step guide, you may look at this blog post

Remember that Core installation is minimal and restrictive. Hence you would need to enable remote administration and file sharing. This can be done either by using netsh commands or PowerShell.

Enable-NetFirewallRule -DisplayGroup "Remote Desktop"
Enable-NetFirewallRule -DisplayGroup "File and Printer Sharing"

netsh advfirewall firewall set rule group=”Remote Desktop” new enable=Yes
netsh advfirewall firewall set rule group=”File and Printer Sharing” new enable=Yes

Once the VM has been installed and configured, convert it to a template. Create a customization specification in vCenter Server and deploy a VM from the template. Before powering on the new VM, resources as demanded by VBO - in a very strict lab environment you could start with 2 vCPU and 4 GB of RAM. 

It's now time to install VBO365. The distribution package is made of 3 msi files:
  • Veeam Backup for Microsoft Office 365
  • Veeam Explorer for Microsoft Exchange
  • Veeam Explorer for Microsoft SharePoint
Since we will install separately the server and the explorers, we need to transfer only server msi file. Copy the file to a share on the virtual machine. Then logon to the server and in cmd prompt change location to where the installer has been copied. To install VBO365 server from command line, run the following command:

msiexec /i "Veeam.Backup365_2.0.0.567.msi" /qn ADDLOCAL=BR_OFFICE365,PS_OFFICE365 /L*V "vbo365.log"

This will install only server (BR_OFFICE365) and PowerShell extension (PS_OFFICE365). The install will not show any prompt (/qn) and it will log everything to vbo365.log file in the same folder with the install file.

During installation monitor the log file. Once the install is completed successfully, you will see the following lines at the end of the file:

MSI (c) (E8:C0) [22:10:14:349]: MainEngineThread is returning 0
=== Verbose logging stopped: 8/29/2018  22:10:14 ===

We need to do two more tasks before moving to installation on admin workstation:
  • check services are runing
  • open firewall ports
To check the services are running, type the following command in PowerShell:

Get-Services "Veeam*" | Format-List

You should see the following output:

If any service is not running, you may enable and start it:

Set-Service -Name "Veeam.Archiver.RestFul.Service" -StartupType Automatic 
Start-Service -Name "Veeam.Archiver.RestFul.Service"

To enable firewall ports for the 3 services, run the following:

New-NetFirewallRule -DisplayName "Open Port 9191" -Direction Inbound -LocalPort 9191 -Protocol TCP -Action Allow
New-NetFirewallRule -DisplayName "Open Port 9194" -Direction Inbound -LocalPort 9194 -Protocol TCP -Action Allow
New-NetFirewallRule -DisplayName "Open Port 4443" -Direction Inbound -LocalPort 4443 -Protocol TCP -Action Allow

It's time to logon to admin workstation where console and explorers will be installed. Transfer all 3 msi files to admin workstation, open a command prompt, change folder to msi path and run the following commands one by one (not all at the same time):

msiexec /i "Veeam.Backup365_2.0.0.567.msi" /qn ADDLOCAL=CONSOLE_OFFICE365,PS_OFFICE365 /L*V "vbo365-console.log"
msiexec /i "VeeamExplorerForExchange_9.6.3.567.msi" /qn ADDLOCAL=BR_EXCHANGEEXPLORER,PS_EXCHANGEEXPLORER /L*V "vbo365-vex.log"
msiexec /i "VeeamExplorerForSharePoint_9.6.3.568.msi" /qn ADDLOCAL=BR_SHAREPOINTEXPLORER,PS_SHAREPOINTEXPLORER /L*V "vbo365-vsp.log"

We are ready to connect to VBO365 and start configuring it. But this, in the next post.

UPDATE 2018/12/4

Updating VBO365 from command line. Latest VBO patch comes in the form of msp file. To update VBO installation you would simply need to copy the file on the server and run the following command:

msiexec /update VBO2.0-KB2765.msp /qb /log patch.log

You can view the log after the install. Remote proxies can be upgraded from the console. 

Saturday, August 18, 2018

Adding Realtek 8111 driver to vSphere 6.7 image

While reinstalling home lab with vSphere 6.7, I was remembered (the hard way) that my on-board NIC is based on Realtek 8111 chipset which is not included in the default vSphere installation media.

I had to go accept the challenge of finding the drivers and creating a new bootable vSphere ISO. Nothing I haven't done before, but since it is not often I've decided to make it a blog post.

First I needed to find the drivers. Using a bit of google foo I found the blog of a long time vExpert (thank you) which has also a collection of drivers. I've downloaded the net55-r8168 offline bundle . From VMware site I've downloaded offline bundle for vSphere 6.7. Placed both of them in the same folder and opened a PowerCLI prompt.

First, create a new software depot using the two bundles:

Add-EsxSoftwareDepot "C:\7_KIT\VMW\", "C:\7_KIT\VMW\"

Next, create a new image profile: see what profiles exist, clone one and change its acceptance level to "community" (because the driver I am about to load is community signed):

New-EsxImageProfile -CloneProfile ESXi-6.7.0-8169922-standard -name ESXi-6.7.0-8169922-standard-RTL8111 -Vendor Razz 
Set-EsxImageProfile -ImageProfile ESXi-6.7.0-8169922-standard-RTL8111 -AcceptanceLevel CommunitySupported

Add the driver to image profile:

Get-EsxSoftwarePackage | Where {$_.Vendor -eq "Realtek"}
Add-EsxSoftwarePackage -ImageProfile ESXi-6.7.0-8169922-standard-RTL8111 -SoftwarePackage net55-r8168

Lastly, generate the vSphere 6.7 ISO containing the driver:

Export-EsxImageProfile -ImageProfile ESXi-6.7.0-8169922-standard-RTL8111 -ExportToIso -filepath C:\7_KIT\VMW\VMware-ESXi-6.7.0-8169922-RTL8111.iso

In one picture, it looks like this:

One more step is needed. Since we have the ISO, we just need to write on a bootable USB. To do this, I've downloaded Rufus, portable version. Run the software, select destination a USB stick (it will be overwritten so better not having any useful data on it), selected the source my new ISO and pressed start.

If during the creation of the bootable stick you are asked to update menu.c32, press YES. After it finishes I've plugged the stick in my physical box and happily installed ESXi.

File download test reached 800 Mbps, a normal value keeping in mind the connectivity between my laptop and the ESXi host.