I ran into an issue recently with the vCenter Server Appliance where the file based backup jobs suddenly stopped working on environments running 6.7. Attempts to run the backup job manually failed with the UI reporting “Location type SMB is not supported”, which is unexpected since SMB is supported.
IIS WMSvc Automated Certificate Management
I’ve added a new script to my GitHub PowerShell repository for managing the IIS WMSvc Certificate. The script WMSvc_InstallCertificate.ps1 is intended to be ran either by scheduled task or by command line and will attempt to detect when the Web Management Service (WMSvc) certificate needs to be replaced. Replacement certificates are sourced from an Enterprise Certificate Authority automatically. The IIS machine account must have privileges to enroll.
In my lab, this script is tied to a scheduled task that is automatically created by group policy on servers attached to the IIS role security group. This same security group is also granted enroll privileges on the certificate template. In effect, new servers created will automatically receive a trusted certificate for their management port and that certificate is rotated automatically before expiration.
Since my lab IIS installations run on Server Core, it is quite convenient to have the remote management service configured automatically.
VMware App Volumes Datastore Migration – 2.18
This is a guide for migrating App Volumes app stacks, metadata, and entitlements to a new datastore. I’ll also cover the clean up steps to remove references to the old location(s). What will not be covered are writable volumes.
These instructions are intended for App Volumes 2.18 and may be applicable to other versions in the 2.x branch. The steps for metadata cleanup were obtained through collaboration with VMware Business Critical Support.
Update: These steps continue to work through at least App Volumes 4.9 (2212).
Requirements
- Administrator role in vSphere vCenter.
- Administrator role in App Volumes Manager.
- Administrator privileges to the App Volumes Manager operating system.
- Access to the App Volumes database server, SQL Server Management Studio, and at least DBO privileges on the App Volumes database.
- A recent database backup in case a rollback is required.
Summary Steps
- Use Storage Groups to duplicate existing app stacks and metadata to a new datastore.
- Remove the Storage Group after duplication has occurred.
- Remove the old datastore.
- Metadata cleanup
vCenter Storage Providers appear Offline after Host Certificates are Installed
I ran into an issue where configuring vSphere with an intermediate signing certificate and replacing certificates on all hosts would cause the storage providers to go offline. Refreshing host certificates would also cause the issue.
VMware support determined the cause was the removal of the vCenter SMS certificate in the hosts local trust store. This certificate is not pushed to the hosts when the trust store was refreshed. This behavior occurs at least on vSphere 6.7 and 7.0. To resolve the issue, it is necessary to manually add the SMS certificate to the hosts trust store.
Local and Domain NTP Overridden by Secure Time Service
Starting with Windows 10 1511, Microsoft introduced a new feature called Secure Time Seeding, part of the Secure Time Service (STS), as an upgrade to the W32TIME service. The STS uses information from SSL connections to validate NTP data. Information from this feature supersedes all other time sources, including locally configured NTP, domain controllers, and Hyper-V time synchronization.
I first noticed the feature when several of my Hyper-V virtual machines began shifting their system clocks backwards and forwards several times a minute. At first the time changes spanned a few hours, but as the machine uptime climbed, so did the time jumps. Eventually, the time was bouncing backwards and forwards by weeks, several times a minute. The Hyper-V time synchronization service was fighting with the new Secure Time Service and this wrecked havoc on authentication and any other services running on the systems.