In Part 1 of “Replace a Failed External PSC in Enhanced Linked Mode”, I worked through repointing a Windows vCenter Server to another External PSC, in an effort to unregister, and rebuild a failed PSC.

In this guide, I will walk through:

• Building a new Windows external PSC
• Joining the SSO Domain
• Re-pointing VC1 back to this newly-built, and linked external PSC, therefore, returning us to the original topology we started with (except for the server name)

The main systems I’ll be working with in this guide are:

• VC1 (Windows vCenter that was re-pointed to working PSC for other site while we rebuild it’s home PSC)
• PSC3 – This is a newly built Windows Server 2012 R2 PSC that is taking the place of PSC1.

I would have chosen to use the same name of the original PSC, but from experience, it’s always better to be safe, and not try to re-introduce a problematic record into the environment, just in case there are still entries hanging out somewhere in the working configuration.

At the end of this guide, we will end up with this topology:

Install the External PSC Role on the New Server

Note: These steps are the same ones to deploy the second PSC in “Deploying Windows vCenter with External PSCs in Enhanced Linked Mode: Part 2.”

1. Launch the vCenter Server Installer
2. Select vCenter Server for Windows, and click Install.
3. Click Next on the welcome screen.
4. Accept the EULA, click Next.
5. Under External Deployment, select Platform Services Controller, and click Next.
6. Verify the system name (This should be the same FQDN of the PSC you are building to replace the failed one), and click Next.
7. Select Join a vCenter Single Sign-On domain.
8. Enter the FQDN for the first Platform Services Controller that owns the SSO domain you want to join.
9. Enter the vCenter Single Sign-On password, then click Next.
10. When prompted for Certificate Validation, click OK to accept the self-signed certificate.
11. Select Join an existing site, choose the site from the dropdown menu(should match the site name of the first PSC you created), and click Next.
12. On the Configure Ports page, make any changes necessary for your environment, and click Next.
13. Set the PSC installation and data directories, and click Next.
14. Select whether or not to join the Customer Experience Improvement Program (CEIP), and click Next.
15. Verify the installation summary settings, and if all looks well, click Install.
16. Once the installation has completed, log into the vSphere Web Client, and navigate to Home > Administration > Deployment > System Configuration.  Under the Nodes object, verify that there are now 4 nodes (you should see 2 PSCs, and 2 vCenter servers).

Next Steps:

After verifying functionality of the newly added PSC, the next step is to re-point VC1 (repointed previously to PSC2) to the new PSC (PSC3).

1. Log onto the vCenter Server instance (VC1).
2. In the command prompt (run as administrator), navigate to C:\Program Files\VMware\vCenter Server\bin (or wherever you have vCenter installed to).
3. Run the cmsso-util script:

   cmsso-util repoint --repoint-psc psc_fqdn_or_static_ip [--dc-port port_number]

   psc_fqdn_or_static_ip – is the FQDN or static IP address of the PSC you want to repoint to.

   

4. Log into the vCenter Server instance by using the vSphere Web Client to verify that the vCenter server is running and can be managed.
5. Finally, to see what PSC each vCenter is connected to:
   1. Log into the vSphere Web Client, and navigate to Hosts and Clusters View.
   2. Select a vCenter, and go to the Manage Tab.
   3. In Settings, go to Advanced Settings.
   4. Search for the config.vpxd.sso.admin.uri
   5. When the result is returned, look at the Value field, and this will tell you what PSC the particular vCenter is connected to.

This completes the series for Replacing a Failed External PSC in Enhanced Linked Mode. If you find that this has helped you, please feel free to share the information. It took me quite a while to gather all the information needed, and build the environment for this, so I really hope it helps.

In my research when first encountering the issues with my failed PSC, I found that there are a lot of other bloggers out there who have written something about the issues, troubleshooting steps, and fixes related to a failed PSC. While this is not a “one fix to rule them all” solution, it is a very clean way to replace a failed PSC. I apologize for not documenting the same thing for the VCSA, however, if you follow the steps in the order I provided, the links I have in my posts also have the proper steps to execute for the VCSA.

If you’ve followed along starting with the 2-part series about “Deploying Windows vCenter with External PSCs in Enhanced Linked Mode…”, this is the next installment, in which we will go through replacing a failed external PSC using that same topology.

If you haven’t followed along, then the following links will help set the stage for what we’re doing here:

• I have never performed this procedure before.
• I would like to know how to do this and be able to help others out by sharing my experience and the information.
• I would like to understand the requirements and general order of operations for performing the procedure.
• I would like to address a current production issue without experimenting in production, and therefore causing further complications or a causing a complete loss of manageability of my environment.

The Workflow

Lab Testing

From PSC1:

.\vdcrepadmin.exe -f showpartnerstatus -h localhost -u administrator -w [password]

.\vdcrepadmin.exe -f showpartnerstatus -h localhost -u administrator -w [password]

If I run the following on PSC2, it will show that PSC1 is now offline:

.\vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w [password]

If I run the following on PSC2, it still shows that there are two PSCs registered:

.\vdcrepadmin -f showservers -h localhost -u administrator -w [password]

Unregister the Failed PSC

1. On the PSC (live one that you just repointed to), open a command prompt (run as administrator).
2. Browse to C:\ProgramData\VMware\vCenterServer\cfg\install-defaults.
3. On the failed PSC: Open the vmdir.ldu-guid file to find the hostid.
4. On the working PSC: Navigate to C:\Program Files\VMware\vCenter Server\bin
5. On the working PSC: Run the cmsso-util unregister command to unregister the stopped/failed Platform Services Controller:

   cmsso-util unregister --hostId host_Id --node-pnid Platform_Services_Controller_System_Name --username administrator@vsphere.local --passwd [password]

   

   After this has been run successfully, verify that the OLD PSC has been removed from the topology.

6. n the vSphere Web Client, navigate to Home > Administration > Deployment > System Configuration.  Under the Nodes object, verify that there are only 3 nodes (you should see 1 PSC, and 2 vCenter servers).
7. On PSC2, run

   .\vdcrepadmin -f showservers -h localhost -u administrator -w ************

   

   You should now only see one server in the listing, as opposed to 2, since you just removed the failed PSC.

8. Delete the failed PSC (VM) you no longer need from the vSphere inventory.

There you have it.  We have successfully re-pointed a vCenter to another PSC, unregistered the bad PSC, and validated that we are now ready to rebuild in order to re-instate the original topology.  The next part to this series will cover building the replacement PSC, joining the SSO domain, and finally, repointing the vCenter at this new PSC, therefore returning the topology to where it was before we started.

If deploying in a production environment, refer to the vSphere Installation and Setup for vSphere 6.0 Guide.

1. Launch the vCenter Server Installer
2. Select vCenter Server for Windows, and click Install.
3. Click Next on the welcome screen.
4. Accept the EULA, click Next.
5. Under External Deployment, select Platform Services Controller, and click Next.
6. Verify the system name (recommended you use FQDN, not IP Address), and click Next.
7. Select Join a vCenter Single Sign-On domain.
8. Enter the FQDN for the first Platform Services Controller that owns the SSO domain you want to join.
9. Enter the vCenter Single Sign-On password, then click Next.
10. When prompted for Certificate Validation, click OK to accept the self-signed certificate.
11. Select Join an existing site, choose the site from the drop-down menu (should match the site name of the first PSC you created), and click Next.
12. On the Configure Ports page, make any changes necessary for your environment, and click Next.
13. Set the PSC installation and data directories, and click Next.
14. Select whether or not to join the Customer Experience Improvement Program (CEIP), and click Next.
15. Verify the installation summary settings, and if all looks well, click Install.

Install vCenter Server and the vCenter Server Components, Connecting to the Second PSC

1. Launch the vCenter installer and select vCenter Server for Windows. Click Install.
2. Once the installer initializes, click Next.
3. Accept the VMware End User License Agreement, and click Next.
4. Under External Deployment, select vCenter Server and click Next.
5. Enter the system’s FQDN, and click Next.
6. Enter the information for the external PSC that was deployed in the section above, and click Next.  This step will register the vCenter with the PSC.
7. When prompted for certificate validation, click OK to approve the self-signed certificate created by the PSC.
8. Configure the vCenter service account according to your environment requirements and click Next.  If you are using an external database server, you will need to specify a user service account.
   
   Note: If you are using a user service account, you will need to make sure it has the “log on as a service” privilege in the local security policy.
   
   
9. Select your database deployment and enter information if necessary, then click Next.
10. Configure the required ports if necessary to match your environment, and click Next.
11. Configure the installation directory for the vCenter Server and data, then click Next.
12. Review all settings, and when ready, click Install.

In many cases for small environments, it makes sense to deploy a vCenter Appliance with an embedded Platform Services Controller.  In larger environments with multiple sites, although you can (doesn’t mean you should) manage remote hosts with a single vCenter, it may make more sense to deploy a vCenter in each site, or region to further increase availability.  Furthermore, by configuring Enhanced Linked Mode, you can make management simpler by being able to link vCenter systems and replicate roles, permissions, licenses, policies, and tags; instead of having vCenter servers which are all managed individually.

Enhanced Linked Mode not only allows you to connect Windows vCenters, or just vCenter Server appliances; you can have an environment where Windows vCenters as well as vCenter Appliances can be linked together.  That said – there are supported and unsupported topologies, which is beyond the scope of this, however, you can find the information for those topologies in this VMware KB Article.

The following procedure is my re-creation of our production topology in a test environment.  The steps will walk through standard installation and configuration of the vCenter servers and PSCs in Enhanced Linked mode to get a to a usable state.  The follow-up post to this will be related to Installing the second PSC and vCenter, linking to the SSO domain and site created in this how-to.

Disclaimer:  The following information and procedures are performed in an isolated environment, and I am building these with the absolute minimum requirements since my test environment is a single physical host that also hosts other test machines in use by another engineer.  If performing these steps in production, be sure to follow proper sizing best practices, and built to suit your particular environment using the resource links provided throughout the article.

Hardware Requirements: vCenter Server for Windows

Before we get into the installation procedure, here’s an idea of what I’m building and the topology I will be using.

(All servers listed below are running Windows Server 2012 R2)

• 1 Active Directory Domain Controller
• 1 Single Sign-On Domain
• 2 Sites (simulated)
• 2 External Platform Services Controllers (one for each site)
• 2 vCenter Servers (one for each site)

Install a Platform Services Controller on a Windows Host

• Verify system meets the minimum HW/SW requirements
• Download the vCenter Server Installer
• Install Adobe Flash Player version 11.9 or later if you will be running using the vSphere Web Client from one of the host machines.
• Forward and Reverse DNS entries should be created for each system prior to installation.

Installation

1. Launch the vCenter installer and select vCenter Server for Windows. Click Install.

   

2. Once the installer initializes, click Next.
3. Accept the VMware End User License Agreement, and click Next.

   

4. Under External Deployment, select Platform Services Controller and click Next.

   

5. Enter the system’s FQDN, and click Next.

   

6. If this is the first PSC being created, create a new Single Sign-On Domain, set the password, provide the site name, and click Next.

   

7. Configure the ports as needed (I left this at the defaults, as I have no need to change them), then click Next.

   

8. Change the installation and data directories as needed and click Next.  For my implementation, I kept the defaults, since this is not production.

   

9. Select whether or not you want to join the customer experience improvement program (CEIP), then click Next. Since this is an isolated environment with no external internet access, I unchecked it.

   

10. At the Ready to Install screen, verify settings, and if everything looks good, click Install.  When done, the installer will provide you with next steps (convenient!)

    

Install vCenter and the vCenter Components

1. Launch the vCenter installer and select vCenter Server for Windows, then click Install.

   

2. Once the installer initializes, click Next.
3. Accept the VMware End User License Agreement, and click Next.

   

4. Under External Deployment, select vCenter Server and click Next.

   

5. Enter the system’s FQDN, and click Next.

   

6. Enter the information for the external PSC that was deployed in the section above, and click Next.  This step will register the vCenter with the PSC.

   

7. When prompted for certificate validation, click OK to approve the self-signed certificate created by the PSC

   

8. Configure the vCenter service account according to your environment requirements and click Next.  If you are using an external database server, you will need to specify a user service account.

   Note: If you are using a user service account, you will need to make sure it has the “log on as a service” privilege in the local security policy.

   

9. Select your database deployment and enter information if necessary, then click Next.

   

10. Configure the required ports if necessary to match your environment, and click Next.

    

11. Configure the installation directory for the vCenter Server and data, then click Next.

    

12. Review all settings, and when ready, click Install.

    

• Configure vCenter to integrate with your Active Domain or LDAP server as needed.
• Configure any group memberships, roles, permissions, licenses, etc…
• Install the next PSC and be sure to join the existing Single Sign-On domain that you just created.  Following that, install the next vCenter, and you are done!