VCF 3.10 Deploying Workload Domains (WLD) via the API

Whilst working on a customer project I had a requirement to deploy new VCF 3.10 Workload Domains on hosts which would use four Physical Nics (PNics) and separate network traffic over two virtual distributed switches (vDS). The SDDC Manager GUI does not support the use of multiple PNics across multiple vDS, and so the deployment has to be performed using the API.

I decided to document the steps I followed, with the JSON payload examples for each step in case it can help to speed up the process for other people. While the VMware documentation outlines the API calls and provides some examples of how to format the JSON I found I had to jump between different sections of the documentation.

This post will cover the creation of a single cluster WLD via the API. In later posts I will cover adding an additional cluster to an existing WLD and also adding additional hosts to an existing cluster within a WLD. I will be using NSX-V as the networking technology, however NSX-T can be used instead and the JSON adapted as per the VMware documentation.

I will use examples of the JSON from the VMware documentation and adapt them to match my use cases, as I can not directly share the information used for my customer.

Prerequisites

Before deploying a WLD via the API you will still need to do the following:

1. Deploy the Management Domain using Cloud Builder
2. Create Network Pools for vMotion and vSAN/NFS for each WLD
3. Prepare the ESXi hosts as per the VCF documentation
4. Add the hosts to SDDC Manager via the Commission Hosts process and assign them to the correct Network Pool either via the GUI or the API.

You will also need the following tools:

1. Tool to edit and display JSON files. I used JSON viewer and Notepad ++ as these were already installed in the customer environment.
2. Tool such as Curl or Postman to submit API calls. I used Curl on the SDDC Manager as Postman was not available within the customer environment.
3. If using Curl on SDDC Manager a tool to upload files to the SDDC Manager appliance such as WinSCP.
4. The credentials of the SDDC Manager REST API User (admin) and the SDDC Manager Super User (vcf).

Creating a WLD

Payload JSON – vDS and PNic association

Reference: Section 2.7.1 of the VCF API documentation https://vdc-download.vmware.com/vmwb-repository/dcr-public/2d4955d7-fb6f-4a61-be78-64d95b951ccd/c6e26ae1-9438-4da0-bfc7-2e21d9046820/index.html#_create_a_domain

When using multiple PNics and vDS per host the sample JSON provided with the api documentation needs to be updated. For each host that is listed in the hostSpecs section of the JSON the hostNetworkSpec section is amended to add a new vmNic entry for each physical adapter. The vdsName is then set to the name of the vDS you want to attach that adapter to, in my customer use case two PNics were to be assigned to each vDS, which gave a hostSpecs section like the example below:

"computeSpec" : {
      "clusterSpecs" : [ {
        "name" : "Cluster1",
        "hostSpecs" : [ {
          "id" : "<Host1_ID>",
          "hostNetworkSpec" : {
            "vmNics" : [ {
              "id" : "vmnic0",
              "vdsName" : "SDDC-Dswitch-Private1"
            }, {
              "id" : "vmnic1",
              "vdsName" : "SDDC-Dswitch-Private0"
            },{
              "id" : "vmnic2",
              "vdsName" : "SDDC-Dswitch-Private1"
            },{
              "id" : "vmnic3",
              "vdsName" : "SDDC-Dswitch-Private0"
            } ]
          }
        },

The second change needed is to the networkSpec section where the types of traffic associated with each vDS instance is specified. For my customer the traffic was to be separated so that all management related traffic used one vDS and the VM traffic would use a second vDS. This gave a networkSpec section like the example below:

"networkSpec" : {
          "vdsSpecs" : [ {
            "name" : "SDDC-Dswitch-Private0",
            "portGroupSpecs" : [ {
              "name" : "SDDC-DPortGroup-Mgmt",
              "transportType" : "MANAGEMENT"
            }, {
              "name" : "SDDC-DPortGroup-VSAN",
              "transportType" : "VSAN"
            }, {
              "name" : "SDDC-DPortGroup-vMotion",
              "transportType" : "VMOTION"
            }]
          },
          {
            "name" : "SDDC-Dswitch-Private1",
            "portGroupSpecs" : [ {
              "name" : "SDDC-DPortGroup-VM",
              "transportType" : "PUBLIC"
            }]
          } ],
          "nsxClusterSpec" : {
            "nsxVClusterSpec" : {
              "vlanId" : 0,
              "vdsNameForVxlanConfig" : "SDDC-Dswitch-Private0"
            }
          }
        }
      } ]
    },

Payload JSON – Gathering the ESXi Host ID values

Reference section 2.5.2 of the VCF API documentation https://vdc-download.vmware.com/vmwb-repository/dcr-public/2d4955d7-fb6f-4a61-be78-64d95b951ccd/c6e26ae1-9438-4da0-bfc7-2e21d9046820/index.html#_usecase_gethosts

Within the payload JSON hostSpecs example in the previous section the id field of each host contained a placeholder value such as <Host1_ID>. The value required for this field is the unique id assigned by SDDC Manager to the ESXi host when it completes the Commission process. The id value is not displayed in the GUI as far as I could locate, and therefore must be retrieved via the API.

Note: This id value does not persist if a host is decommissioned and then added back to SDDC Manger.

To get the list of hosts you can perform a GET operation on SDDC Manager, however it is worth using a filtered API call to query just for hosts marked as having a status of UNASSIGNED_USEABLE. This status means a host is available for use and can be assigned to a new WLD, this filters out any hosts already assigned to WLDs.

The API url to use is

<SDDC Manager FQDN>/v1/hosts?status=UNASSIGNED_USABLE

When using curl on the SDDC Manager appliance I like to send the output to a file so that I can view it using a JSON viewer rather than as a single stream of text. This is also useful if I have multiple WLDs or tasks to complete and will need different hosts from SDDC Manager each time. The command to run is:

curl "https://<SDDC Manager FQDN>/v1/hosts?status=UNASSIGNED_USABLE" -i -u "admin:<SDDC Manager API User Password>" -X GET -H "Accept: application/json" > <json output file name>

The output of the command contains details about the specification of each host matching the status filter. If you remove the header text:

HTTP/1.1 200 OK 
Content-Type: application/json 
Content-Length: 1424

from the file you can open it with a standard JSON viewer. The values you are interested in are the id fields of each host. Expand the elements section and each numbered entry represents a single host. Locate the hosts you want to use for the new WLD and copy the value from the id field. Paste the value into the payload JSON for the new WLD to replace the appropriate placeholder value.

Validating the Payload JSON

Reference section 2.7.1 of the VCF API documentation https://vdc-download.vmware.com/vmwb-repository/dcr-public/2d4955d7-fb6f-4a61-be78-64d95b951ccd/c6e26ae1-9438-4da0-bfc7-2e21d9046820/index.html#_create_a_domain

Fill in the remaining values in the JSON such as passwords, IP addresses and FQDNs for the components that will be deployed in the WLD and your JSON is ready for validation.

If you can check the JSON structure is valid first using a JSON formatter or viewer, this eliminates missing quotes or curly braces. When you have a valid JSON file upload it to the SDDC Manager appliance (if using curl on the appliance) using the vcf user account.

To validate that the JSON file matches the expected format for SDDC Manager it can be submitted via an API call using the URL

<SDDC Manager FQDN>/v1/domains/validations

The full curl command to use, parsing the JSON from a file rather than on the command line is

curl "https://<SDDC Manager FQDN>/v1/domains/validations" -i -u "admin:<SDDC Manager API User Password>" -X POST -H "Accept: application/json" -d @ <json input file name>

If the command returns an executionStatus value of COMPLETED and a resultStatus value of SUCCEEDED the JSON is a valid domain spec file. In the case of an error it will highlight in the error message the cause of the failure, such as an expected field is missing.

Submitting the Payload JSON

Reference section 2.7.1 of the VCF API documentation https://vdc-download.vmware.com/vmwb-repository/dcr-public/2d4955d7-fb6f-4a61-be78-64d95b951ccd/c6e26ae1-9438-4da0-bfc7-2e21d9046820/index.html#_create_a_domain

The final stage of the WLD creation process is to submit the validated JSON to create the task for the creation of the WLD. Before making the final API call the JSON file used for the validation API call needs to be modified. Although it is not explicitly called out in the API documentation as a step to perform if you compare the JSON spec used for the validation and the submission of the task you will notice they are slightly different. The JSON spec used for the WLD creation task does not contain the domainCreationSpec field at the start of the file.

Using the example spec from the API documentation for the validation stage the start of the file looks like this

{ "domainCreationSpec" : 
{
    "domainName" : "Domain1",
    "vcenterSpec" : {
      "name" : "vcenter-1",
      "networkDetailsSpec" : {
        "ipAddress" : "10.0.0.43",
        "dnsName" : "vcenter-2.vrack.vsphere.local",
        "gateway" : "10.0.0.250",
        "subnetMask" : "255.255.255.0"
      },

When editing the file for the WLD creation task remove the first line of the file so that it now starts like this

{
    "domainName" : "Domain1",
    "vcenterSpec" : {
      "name" : "vcenter-1",
      "networkDetailsSpec" : {
        "ipAddress" : "10.0.0.43",
        "dnsName" : "vcenter-2.vrack.vsphere.local",
        "gateway" : "10.0.0.250",
        "subnetMask" : "255.255.255.0"
      },

At the end of the file remove the last curly brace ( } ) character as well which originally closed the domainCreationSpec section of the JSON. The file should now end with two close curly brace characters like this

"nsxVControllerSpec" : {
        "nsxControllerIps" : [ "10.0.0.45", "10.0.0.46", "10.0.0.47" ],
        "nsxControllerPassword" : "Test123456$%",
        "nsxControllerGateway" : "10.0.0.250",
        "nsxControllerSubnetMask" : "255.255.255.0"
      },
      "licenseKey" : "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX",
      "nsxManagerAdminPassword" : "VMware123!",
      "nsxManagerEnablePassword" : "VMware123!"
    }
  }

Upload the new JSON file to the SDDC Manager if using curl, and submit the request using the following command

curl "https://<SDDC Manager FQDN>/v1/domains" -i -u "admin:<SDDC Manager API User Password>" -X POST -H "Accept: application/json" -d @ <json input file name>

The response will include an id field, which is the task id for the submitted WLD creation task. You can monitor the progress of the task via the SDDC GUI if you have access, or you can use the task id from the output and monitor it via the api using the command

curl "https://<SDDC Manager FQDN>/v1/tasks/<Task ID>" -i -u "admin:<SDDC Manager API User Password>" -X GET -H "Accept: application/json" 

If the task fails and you need to resubmit it using the same JSON spec as before you can use the command

curl "https://<SDDC Manager FQDN>/v1/tasks/<Task ID>" -i -u "admin:<SDDC Manager API User Password>" -X PATCH -H "Accept: application/json"

In my next post I’ll cover the process of adding new clusters to an existing WLD using the API.