Credit to Gary Blake who documented this originally in 2020 and pointed readers at an official KB article for VCF 3.x.
In my case, I hit an issue with VCF 4.3 and found that the fix was still valid. I wanted to point out one additional validation step. This is not something that Gary missed in his article, I just didn’t initially spot this when I went through updating the json and so had to run through the fix a second time. I thought it would be worth documenting it in case others do the same, and also as a reminder to myself the next time I run across this issue.
When performing a workload domain creation task via an API call (you could also be using the GUI and this would be valid) the json specification I had provided contained license keys for vSphere and for vSAN which did not have sufficient capacity for the number of hosts I was using. I should have noticed this before submitting, but I didn’t and the WLD creation task failed at the step to apply the licenses.
For vSphere I was able to simply update the license key on each of the ESXi hosts, as my json spec did not specify the license key to use for the ESXi hosts. Restarting the task in SDDC Manager it moved and then failed at the vSAN license stage. For this step, I was not able to work around it by simply adding the license key to the vCenter server. That’s when I found Gary’s article.
The procedure in the article is still valid:
- Log in to the SDDC Manager GUI and collect the Task Id by selecting the failed task in the list to view the subtasks and copying the ID value that is appended to the URL in the browser.
- Log in to the SDDC Manager via ssh and switch to the root user.
- Run a curl command to retrieve the json spec for the task (replace <Task_ID> with the task id you collected in step 1 and remove the -k if you have a trusted SSL certificate on the SDDC Manager appliance)
curl -k -s http://localhost/domainmanager/internal/vault/<Task_ID>| json_pp > /tmp/spec.json
- Edit the json file using vi on the SDDC Manager appliance, or if you prefer download it using a tool such as WinSCP to your local machine (you will need to change the owner and permissions to the vcf user first as it does not have permissions by default)
- Find the value you need to update, replace it and save the file. Important: In my case the vSAN license key appeared in five different places within the json file. Under the main json spec where I detailed the vsanDatastoreSpec values and then four more times under a section called addViUberSpec. You need to update all references to the old value or the task will still fail.
- Run a second curl command to update the spec within the task itself in SDDC Manager (replace <Task_ID> with the task id you collected in step 1 and remove the -k if you have a trusted SSL certificate on the SDDC Manager appliance)
curl -k -H 'Content-Type:text/plain' -X PUT http://localhost/domainmanager/internal/vault/<Task_Id> -d @/tmp/spec.jsonNote: There is no confirmation message provided by the above command and SDDC Manager does not restart any services.
- You can now restart the failed task in SDDC Manager GUI and it will restart using the updated spec. If you have correctly updated the values it should now successfully complete the sub task that previously failed.
You can use this process multiple times with a single failed task to update values if you encounter subsequent failures.