Today i ran into an issue with an setup of NSX-V version 6.4.1. As this is the second time i run into this error let’s document it for future purpose.

Issue:

Viewing logical switches in the vSphere Web Client displays the error, in vCenter on the Networking & Security Tab under Logical Switches the list was empty and the UI gave the following Internal Error:

Ah quick google search pointed us to the following Knowledge Base article:
https://kb.vmware.com/s/article/54442?lang=en_US

 This issue occurs due to stale entries in the NSX Manager database, the NSX database will be inconsistent with the vCenter Server when a VM becomes a template in vCenter Server and there is config changes on the VM template. Any changes to the VM templates are not synced with NSX unless they are converted back to VM. 

NOTE:
Please contact GSS when you run into this issue on a Production environment. And ofcourse if you still run Production on NSX-V you need to take action and move to NSX-T!

Workaround

We need to identify the templates which are causing this issue and then convert the template back to a VM to sync latest updates to NSX inventory. This will remove stale entries in DB and get latest sync from VC, making NSX DB consistent with VC and there by resolving the issue. You can then convert back the VM to a template. You can then convert back the VM to a template.

Login to the NSX Manager go to Enable Mode, fill in the password and go to the engineering mode by typing the “st eng” command. Type y to continue and fill in the password: IAmOnThePhoneWithTechSupport

Now you are in the engineering mode! Got to the database cli by entering “psql -U secureall
You can do the check in 2 ways, the first one is a bit more clicking and checking, but will work when you don’t have loads of templates.

Check 1:

To get the Templates which are causing the issue run the following query:

select objectid, host_id from domain_object where dtype='VimVirtualMachine' and host_id NOT IN (select objectid from domain_object where dtype='VimHostSystem');

You will get a list with objectids which you can check in vCenter by going through the list of templates and check the url for the objectid. If the object id is identical, hen convert the template back to a VM to sync latest updates to NSX inventory and t latest sync from VC, making NSX DB consistent with VC and there by resolving the issue. You can then convert back the VM to a template. 

Check 2:

If you have a lot of Templates you can use to following query to check both objectid and template name:

select * from domain_object where dtype='VimVirtualMachine' and host_id NOT IN (select objectid from domain_object where dtype='VimHostSystem');

and scroll through the output searching for .vmx and vm-####

If you ran through all objectids which were in the list, you can check if the list is empty by running the check command again:

All issues are resolved! Let’s check if we can see the virtual wires again:

Hooray!!!

Leave a comment

Your email address will not be published. Required fields are marked *