Tagging VMs in NSX-T using vRA and vRO

This is a write up of a recent project I did for a customer. The use case was to automatically apply NSX-T tags to VMs deployed via vRA in preparation for micro-segmentation at a later date. This customer had a particular network design that needed to be replicated within the NSX-T environment, and therefore needed the tags within NSX to be determined based on a value on the vRA request form.

Solution Requirements

The customer had the following requirements:

1. Allow the administrators submitting the request in vRA to select the network segment that the VM would be deployed into.
2. Automatically add a Tag to the VM representing the security zone the VM would be deployed into and a second Tag representing the network tier the VM would be connected to. Both values should be determined from the network segment selected on the vRA request form.
3. Ensure the Tags were applied to the VM prior to the customisation spec being applied to the VM in vCenter. This would ensure the correct security policies were applied within NSX-T to allow the VM to communicate with Active Directory, and therefore join the domain as part of the customisation spec.
4. Ensure any Tags already applied to the VM, or automatically applied by the vRA provisioning process were not overwritten.

The customer was running vRA 7.5 and NSX-T 2.3 as part of a VMware Cloud Foundation (VCF) deployment with a single workload domain initially, with plans to expand in the future.

Solution Configuration

NSX-T

As there is no NSX-T plugin for vRO the NSX-T api was used to automate the application of the Tags onto the VMs. The api is well documented on the VMware Code site. Two api calls were used in the solution:

1. List Virtual Machines
2. Update Tags applied to a Virtual Machine

The List Virtual Machines call was used to meet requirement 4, to retrieve a list of the Tags already applied to the VM (if any) from NSX. As the update operation overwrites any Tag values with the values specified in the REST operation the existing Tag values needed to be retrieved and added to the REST operation to ensure they were preserved.

The List Virtual Machines operation can be filtered to return just a single VM in the results, by specifying the name of the VM in the REST operation url. As detailed in the API documentation it is also possible to specify which values should be included within the operation response. For this solution the tags and external_id fields would be included, both of which would be required for the Update Tags operation.

The REST operation used has an input parameter set for the display_name value, which is referred to as vm_name. This configuration within vRO allows the required VM name value to be passed into the REST operation when the workflow is run.

GET https://<nsx manager fqdn>/api/v1/fabric/virtual-machines?display_name={vm_name}&included_fields=tags&included_fields=external_id

The results of the api call look like the following example:

{
  "results": [
       {
          "external_id": "50250362-8683-2850-df49-884b2a4dfa51"
          "tags": [
              {
                  "scope": "",
                  "tag": "app"
              },
              {
                  "scope": "",
                  "tag": "managed_vm"
              }
          ]
        }
  ],
  "result_count": 1
}

For the Update Tags operation the API documentation details that the operation requires a JSON format body to be passed with the operation. The body must specify the external_id of the VM, identifying the VM on which to perform the operation. This value is the vCenter UUID of the VM which can be retrieved from the vRA payload. As a sanity check before updating the VM the value from vRA is checked against the value retrieved from NSX using the List Virtual Machines operation. The body must also contain an array of Tags specifying the name and scope for each Tag.

The REST operation does not require any additional parameters in the url, however a header of Content-Type application/json should be set.

POST https://<nsx manager fqdn>/api/v1/fabric/virtual-machines?action=update_tags

An example of the content body provided with the REST operation is listed in the api documentation. Remembering that the api operation will overwrite the existing tags, the original tags on the VM must be included in the content body along with the new tag to be applied:

{
  "external_id": "50250362-8683-2850-df49-884b2a4dfa51",
  "tags": [
      {"scope": "", "tag": "app"},
      {"scope": "", "tag": "managed_vm"},
      {"scope": "", "tag": "dmz_vm"}
  ]
}

If the request is successful a 204 (no content) status code is returned.

vRA

To ensure that the solution met requirement 3 an event broker subscription set on the Clone Workflow>Customize Machine>Pre lifecycle state was used. Since the Update Tags operation needs the VM object to be present in vCenter (for the object to be present within the NSX-T inventory), the lifecycle state selected needed to be after the VM object was created in vCenter and before the customisation of the machine was started.

To allow users to select the network segment into which to deploy the VM on the vRA request form, a custom property was used. For this solution the network segment was a direct mapping to the Network Adapter value within the Reservation. Therefore the VirtualMachine.Network0.Name system custom property could be used on the request form. To populate the list of values for the drop down on the form, the property was set to External Values and linked to the action getApplicableNetworks (in the com.vmware.vra.networks module within vRO).

vRO

The vRO configuration is fairly simple.

The NSX-T manager is added as a REST endpoint with basic authentication. The username and password specified for the connection must have CRUD (Create, Read, Update and Delete) permissions in NSX-T.

Two REST operations are configured on the endpoint, as outlined in the NSX-T section and workflows are created from these operations to allow them to be executed.

A wrapper workflow was created to perform the retrieval of the VM name, VM UUID and network segment property values from the vRA payload. The network segment value was then used to determine the security zone and network tier values. For this customer the name of each network segment was based on a standard naming format which contains both the security zone and network tier names. This meant those values could be retrieved by splitting the network segment value into multiple strings, of preconfigured lengths.

A custom decision was used to sanity check that the VM external id retrieved from NSX-T via the REST operation matched the value retrieved from the vRA payload, just in case two VMs were ever present within the vCenter with the same name.

REST Operation results

When you execute a REST operation within vRO, the body of the response is stored as a string value named contentAsString. As shown in the NSX-T section earlier, the response is a JSON object, so the contentAsString value will be this JSON object written as a string.

To make navigating the results, and extracting values easier the contentAsString value can be converted back into a JSON object using the JSON.parse(contentAsString) command. It is then possible to extract the external_id value, for example using the dot notation:

var json = JSON.parse(contentAsString);
var externalId = json.results[0].external_id;

Notice that the results part of the JSON object is an array, therefore we must specify which entry within the array (the first entry, therefore 0) that we want to retrieve the external_id value from.

This approach is also used to retrieve the current Tag values from the results. The tag entry in the results is an array of Tag values, which are again in JSON format. As the second REST operation also requires an array of Tag values, the new Tag values can be appended to the array retrieved via the first REST operation. To append to the array the string value of

{"scope": "", "tag": "dmz_vm"}

Can be parsed to JSON (JSON.parse()) and then pushed into the array.

For this solution it was discovered that Tags are not present on the VM, if no NSX Tags have been explicitly set as part of the vRA blueprint when the Clone Workflow>Customize Machine>Pre lifecycle state was reached. In this scenario, the above approach to push the new tag value into the array will fail. This is due to the fact that when no Tags are present on the VM the tags parameter is not included within the results of the first REST call. This in turn results in the array of existing Tags being set to a Null value, and it is not possible for vRO to push a new entry into a Null value.

To allow for this scenario, whilst also allowing any Tags which are present to be included the code to retrieve the Tag values from the results JSON was updated to the following:

var json = JSON.parse(contentAsString);
tagValues = json.results[0].tags||new Array();

This results in a new Array being initialised if no Tag values are found within the results. Then an if statement is used to determine whether the new Tag values need to be added to the array in JSON format or as a single string, based on whether a new array was created. This is due to the fact that vRO throws an exception if you try to store values in different formats within the array.

if(tagValues.length > 1){
  //existing array
  tagValue.push(JSON.parse(newTag);
}
else{
  //new array
  tagValue.push(newTag);
}

To make it easier to create the entry for the new Tag the JSON value (in string format) needed for the new Tags were added to vRO as the value for an attribute. Placeholders are left within the attribute value for the name and scope of the Tag. A single string.replace() command is used to find and replace the name and scope placeholders with the correct values.

Finally once the external id of the VM and the Tag values have been correctly updated within the content body the JSON object is converted back to a string using the JSON.stringify() command and passed to the content attribute of the second REST operation.

At the end of the vRA provisioning process, an additional Tag is added to the VM if an NSX-T backed network is used for the VM. This Tag does not overwrite the custom Tags added to the VM through the event broker subscription workflow.