HOWTO: Replace a failed 3Par drive

HPE 3Pars are great arrays, but just like any other storage system, they do occasionally end up suffering a failed hard drive.  Replacing a failed 3Par drive isn’t quite the same as replacing a failed Proliant Smart Array controller drive – there are a few manual steps that need done to facilitate the replacement process, which I am going to detail below (note – I’m using a StoreServ 7200, based on OS 3.2.1 MU2 as my reference in this post).

First, SSH (via Putty) the 3PAR’s management IP and login as 3paradm (remember the username and password are case-sensitive).

At the 3PAR_SN# cli% prompt, type:    showpd -failed -degraded

This should show you the failed drive and it’s ID (in the example below, the drive hasn’t totally failed, but rather is just degraded due to an internal loop error in the drive, so it needs replaced).

2016.05.24 - 09.15.46 - SNAGIT -  0026

Next, see if servicemag has been issued or is running with:   servicemag status

If servicemag is not running, you will see:   No servicemag operations logged.

Now we want to see if the data has been evacuated off the drive already by running this command:   showpd -space 15   (where 15 is the drive ID that needs replaced).   Using the output shown below, double check there is no data left on the drive. You need to check that all columns other than size and failed are zero.  As you can see from the example , this drive still has data on it (again because the drive in this example is only degraded, not failed – my experience is that typically failed drives have 0, 0, 0, 0 for volume, spare, free, and unavailable, while failed is usually equal to the size).

2016.05.24 - 09.15.54 - SNAGIT -  0027

To evacuate the data, run this command:    servicemag start -pdid 15     and answer yes when prompted if you are sure you want to run it.

2016.05.24 - 09.56.05 - SNAGIT -  0033

To check the status / progress of the servicemag command, run:    servicemag status

2016.05.24 - 09.16.14 - SNAGIT -  0029

As you can see above, 4 chunklets (1GB blocks of disk space) have been moved off the drive so far, with another 107 chunklets (107 GB) to evacuate.  Below is what you will see once the servicemag process has finished.

2016.05.24 - 09.16.23 - SNAGIT -  0030

Before continuing, verify there is no data left on the drive by running:  showpd -space 15

2016.05.24 - 09.16.28 - SNAGIT -  0031

When the HPE field engineer arrives onsite with the replacement disk, you may need to turn on the locate light on the failed drive for him.  To do this, run:      locatecage -t XX cageY ZZ    where TT is time in seconds (i.e. 300), and Y in cageY is the cage number shown above, and ZZ is the magazine number to locate (i.e.  locatecage -t 300 cage0 15 enables the flashing locate light for 5 minutes for the failed drive that is being referenced in this HOWTO).

Once the drive has been replaced, the 3Par **should in the background** run an admitpd automatically for you.  To verify this, run:   showpd -p -mg ZZ -c Y     to see if the new drive is listed (note it will most likely have different drive ID than the dead drive)

When you have verified the new drive has been seen and admitted, you can check the rebuild status with servicemag statusYou can see below the rebuild process, followed by the status message once servicemag as successfully finished.

2016.05.24 - 09.16.52 - SNAGIT -  0032

If you go back to the HP 3PAR Management Console and refresh the console, you should find the fail drive no longer appears (it will stay there appearing as failed even after it has been removed from the cage until the rebuild process is completed, at which point it will go away).

If the HP 3PAR Management Console indicates a firmware update needs performed on the replacement drive, run:   upgradepd ZZ    and answer yes when prompted.  Refresh the HP 3PAR Management Console when the upgrade is complete to check for any other errors.

If no further errors appear, the drive replacement process is completed.  If there are errors, then escalate back to HPE with your original case number.

As always – Use any tips, tricks, or scripts I post at your own risk.

HOWTO: Configure FileZilla Server to use SSL/TLS with a wilcard SSL certificate

Every so often, we have the need to securely transfer large files between very remote systems, and the simplest way to do this is via FTP with the FileZilla client.  Of course the FTP protocol by itself isn’t secure, but if you enable FTPES, it generally is.  FTPES however requires a SSL certificate, and while you could allow the FileZilla server to generate it’s own self signed certificate, we like to use our existing wildcard SSL certificate which requires a few extra steps, as detailed below.

  1. Download the current version of FileZilla Server.
  2. Perform a Standard install accepting all defaults.
  3. Copy the domain’s wildcard certificate in PEM format (i.e. wildcard.jbgeek.net.pem) and the wildcard’s key file (i.e. wildcard.jbgeek.net.key) to “C:\Program Files (x86)\FileZilla Server”.      (see HOWTO: Generate and self-sign Wildcard SSL certs in Ubuntu if you need to convert your existing PFX wildcard certificate to PEM format).
  4. Open Settings in the FileZilla Server management interface and navigate to SSL/TLS settings.
  5. Select “Enable FTP of SSL/TLS support”.
  6. Browse and select the key file you copied above for the Private Key file.
  7. Browse and select the PEM file you copied above for the Certificate file.
  8. Select “Force PROT P to encrypt file transfers in SSL/TLS mode”.
  9. Adjust any other options as required (i.e. password protecting the management interface until “Admin Interface Settings” or enabling MODE Z support under File Transfer compression).
  10. Select Users from the Edit drop down menu and create your users and access rights as required.
  11. Stop and restart the FileZilla Server service.
  12. Open the FileZilla FTP Client, and connect to via FTPES to the FQDN of the FileZilla Server  (i.e. ftpes://myftpesserver.jbgeek.net, along with the username and password combination you just created).
  13. If you configured SSL/TLS correctly using the above steps, the FileZilla FTP Client should prompt you if you wish to trust this certificate – select “Always trust certificate for future sessions” and click ok.

Keep in mind you may need to adjust the ports on your firewalls to allow connectivity, which is outside the scope of this post.

As always – Use any tips, tricks, or scripts I post at your own risk.

HOWTO: Turn on a HDD UID on a HPE Proliant in VMware with HPSSACLI

This morning we needed to replace a hard drive in a HPE Proliant running VMware ESXi at a remote site that had a PFA on it.  Unfortunately, while ILO is great at identifying the defective drive, it has no ability to enable the UID on the drive, and given that this unit is at a remote site, we had no way of knowing in advanced if the fault light was actually turn on for this drive before the HPE field engineering arrived to swap the drive.  So after digging through the help documentation, I found the necessary HPSSACLI command to enable the drive’s UID.

First, to get a list of all the physical drives in an ESXi host, SSH the host and run this command:

/opt/hp/hpssacli/bin/hpssacli ctrl slot=0 physicaldrive all show

This should output a list of all the drives in the system as shown below.

2016.05.19 - 10.14.13 - SNAGIT -  0005

Next, to enable the blue UID LED for 1 hour on port 2I, box 1, bay 2, run this command:

/opt/hp/hpssacli/bin/hpssacli ctrl slot=0 physicaldrive 2I:1:8 modify led=on duration=3600

The blue UID should now come on for 1 hour and then shut off on it’s own.  If you want want to manually shut if off before the 1 hour is up, run the same command again, but change the “led=on” to “led=off”.

As always – Use any tips, tricks, or scripts I post at your own risk.

Upgrade a stuck ILO firmware via SSH

We have had a rash of issues where by upgrading ILO firmware via the WebUI has been failing.  It looks like it finishes, but when you log back in, it is still the original firmware from when you started the upgraded.  And no matter what you do via the WebUI, it just will not upgrade.  So to upgrade the stubborn firmware, the simplest thing to do is SSH the ILO directly and upload the firmware via the console interface.  Below are the steps to do this.

First, you need a running web server to pull the firmware from.  IIS is usually the handiest, so it is simply a matter of adding a mime-type for the binary firmware file.  Open an administrative command prompt and run:

c:\windows\system32\inetsrv\appcmd.exe set config /section:staticContent /+"[fileExtension='.bin',mimeType='application/x-bin']"
iisreset /restart

Extract the ILO firmware bin with 7-Zip and put the bin somewhere within IIS that you can download it.   Next – to save myself extra grief, I also make sure I can actually download the firmware to a regular PC with a browser before continuing.  So open the browser of your choice and make sure you can download the bin to your PC before continuing.

Putty the ILO interface, accepting the SSH key (if prompted), and login.  Once logged in, check, then download the new firmware with the following commands.

*** Note – the ILO will automatically reboot once it successfully downloads the firmware and does not give any indication of the reboot.  As a result, you might want to start a continuous ping to the ILO to see once it has rebooted and is back up ***

show /map1/firmware1
cd /map1/firmware1
load -source http://http_server_ip/ilox_xxx.bin

Once the ILO reboots, you should have a working ILO with the firmware version you want / need.

As always – Use any tips, tricks, or scripts I post at your own risk.

2016.05.12 - 19.34.52 - SNAGIT -  0097

Factory Reset a HPE FlexFabric 5700 to defaults

Not to long ago, we received a new HPE FlexFabric 5700 switch and we proceeded to muck around with the configuration settings trying a few things that we normally would never do to a production switch.  When we were done having fun and learning, we needed to reset the unit back to defaults so we could really deploy it into production.  Of course, resetting a switch to factory defaults is not something you do very often, so we had to actually RTFM.  I’ll save you the time of that here…

From the serial console, execute these commands:

restore factory-default
yes
save
yes
{hit enter}
reboot

When the switch reboots, it will be at defaults.

Below is a screen snapshot of what you’ll see during this process.

2016.05.11 - 14.43.21 - SNAGIT -  0066

HOWTO: Scheduled a standalone VMware ESXi Host Reboot via Powershell

We have several clients who have standalone VMware ESXi hosts (that are not part of any vCenter) without any option for vMotion or Storage vMotion.  This can make it difficult for us to keep those hosts current with patches, updates, and BIOS / firmware because it means we need to manually shut the hosts’ guest down, and then restart the host – none of which can be done during normal business hours – and I’m getting too old to work all night.

Fortunately, VMware provides us a way to use PowerShell to shutdown a ESXi host’s guest, and then force a reboot.  This means we can apply patches and updates late in the day to the ESXi host, then schedule the host to reboot early in the morning after the daily backup completes.  Then when we come into the office in the morning (usually an hour or two before the clients arrive at their offices), it is simply a matter of checking the host to ensure it is back up along with all it’s guests.

To schedule a standalone VMware Host reboot, the current VMware PowerCLI client needs to be installed on the machine that will be running the scheduled reboot.

Once the VMware PowerCLI is installed, you need to create 3 files:

  • C:\WINDOWS\VMWARE_ROOT.PWD – encrypted file that contains the root user’s password
  • C:\WINDOWS\VMWARE_HOST_REBOOT.CMD – the wrapper that will call PowerShell from TaskScheduler
  • C:\WINDOWS\VMWARE_HOST_REBOOT.PS1 – the actual PowerShell script that executes the reboot

To create the file C:\WINDOWS\VMWARE_ROOT.PWD, open PowerShell and run the following command:

read-host -assecurestring "Enter Password" | convertfrom-securestring | out-file C:\WINDOWS\VMWARE_ROOT.PWD

 

At the “Enter Password” prompt, enter the password of the root user account for the ESXi host you want to reboot.

You also need to set the PowerShell Execution Policy to support remote signed scripts such as C:\WINDOWS\VMWARE_HOST_REBOOT.PS1.  To do this, in PowerShell run the following command and select Yes when prompted:

Set-ExecutionPolicy RemoteSigned

We need to schedule a time for VMWARE_HOST_REBOOT.CMD to run.  I’ve set 4:15 am local time on March 22, 2015 in the example shown below, but you can adjust as required.  In an administrative command prompt, run this (***note – this will create the scheduled task to run as the currently logged in user***):

schtasks /create /tn "VMware Host Reboot" /tr C:\WINDOWS\VMWARE_HOST_REBOOT.CMD /sc once /st 04:15:00 /sd 03/22/2015 /rp "*" /ru "%userdomain%\%username%"

Now we need to create C:\WINDOWS\VMWARE_HOST_REBOOT.CMD, which is the batch file task scheduler uses to launch our PowerShell script.

rem --- begin cut and paste of notepad C:\WINDOWS\VMWARE_HOST_REBOOT.CMD
@echo off
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy RemoteSigned -noprofile -File C:\WINDOWS\VMWARE_HOST_REBOOT.PS1
exit /b
rem --- end cut and paste of C:\WINDOWS\VMWARE_HOST_REBOOT.CMD ---

Lastly, we need to create C:\WINDOWS\VMWARE_HOST_REBOOT.PS1, adjusting the variable for $server to the host you wish to reboot (all variables are all defined at the top of the script) and adjust wait time ($waittime) before force rebooting after you issue a graceful gust shutdown command.

###--- begin cut and paste of notepad C:\WINDOWS\VMWARE_HOST_REBOOT.PS1
### VMWARE_HOST_REBOOT.PS1
### @deancolpitts – http://blog.jbgeek.net
### 2015.03.20

### This script will attempt to perform a graceful VM restart via the VMware Tools inside the guest.
### Variables - please only adjust server, user, and waittime. Any other variables should not be touched.
### Server is the vCenter server or ESXi host's FQDN, while user is the vCenter user or ESXi user account.

$server = "VMWARE.FQDN.DOMAIN_OR_IPADDRESS"
$user = "root"
$waittime = "300"

$credentialFile = "C:\WINDOWS\VMWARE_ROOT.PWD"
$pass = cat $credentialFile | convertto-securestring
$credentials = new-object -typename System.Management.Automation.PSCredential -argumentlist $user,$pass

add-pssnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue -WarningAction SilentlyContinue | Out-Null
if ( $DefaultVIServers.Length -lt 1 )
{
Connect-VIServer -Server $server -Protocol https -credential $credentials -WarningAction SilentlyContinue | Out-Null
}

Get-VM | Shutdown-VMGuest -confirm:$false -WarningAction SilentlyContinue

### Wait x number of seconds for all the VM's to gracefully shutdown before a forced kill occurs
Start-Sleep -s $waittime

Restart-VMHost -VMHost $server -force -confirm:$false

###--- end cut and paste of C:\WINDOWS\VMWARE_HOST_REBOOT.PS1 ---

All that is left do now is wait for C:\WINDOWS\VMWARE_HOST_REBOOT.CMD to run at your scheduled time.

As always – Use any tips, tricks, or scripts I post at your own risk.

KB3148812 breaks Windows Server Update Services

Earlier this week, Microsoft pushed out KB3148812, which enables ESD decryption provisioning in WSUS (on Windows Server 2012 and Windows Server 2012 R2).  From what I read and understand, KB3148812 is going to be a mandatory update for WSUS to support Windows 10 updates after May 1.  Unfortunately, it appears that KB3148812 also breaks the WSUS console.  Rather than re-issue KB3148812 so it doesn’t break WSUS, Microsoft has published in a blog posting the necessary post-install steps to “un-break” WSUS after install KB3148812.

Basically you need to perform two steps to return WSUS to a working condition on Windows 2012.  First, you need to re-run the WSUS post-install.  And then you also need to add HTTP Activation to your WSUS server.

For Step 1, to re-run post-install, open an administrative command prompt and run:

"C:\Program Files\Update Services\Tools\Wsusutil.exe" postinstall /servicing

For Step 2, to install HTTP Activation, open an administrative PowerShell command prompt and run:

Install-WindowsFeature -Name NET-WCF-HTTP-Activation45

No reboot should be necessary and the WSUS console should now open and function normally for you.  There is the possibility you’ll still get client connectivity issue though with an error of 0x80244007, which is something entirely different that Microsoft is still looking at as of 2016.04.22.

HOWTO: Monitor the rebuild status of a HPE SmartArray in ESXi 5.5

To monitor the rebuild status of a HP SmartArray controller in VMware ESXi 5.5, you need to have the HP VMware tools bundle installed (which is installed if the server was installed from the HP VMware media / ISO).  Once the tools bundle has been installed, simply SSH the server (or go right on the console, either physically or via ILO), login and run:

/opt/hp/hpssacli/bin/hpssacli ctrl all show status

This will provide you a list of all the SmartArray controllers in the server.  From this list, find the slow number of the controller that contains the logical drive you need to check the status on and run the following command (substitute slot=XX for the slot value you determined with the previous command):

/opt/hp/hpssacli/bin/hpssacli ctrl slot=XX ld all show

2016.04.14 - 09.12.11 - SNAGIT -  0000

If you happen to running an older version of ESXi 5.x, or your HP VMware Tools bundle is not somewhat recent, then the commands are somewhat different.  In this case the correct commands are:

/opt/hp/hpacucli/bin/hpacucli
ctrl all show
ctrl slot=0 ld all show

HPE Insight Remote Support 7.6 auto-upgrade fails

As some of you may have noticed, HPE rehpe_pri_grn_pos_rgbleased Insight Remote Support (IRS) version 7.6 this week.  Among other things, the interface is now rebranded with the new HPE logo and icon, it has better security logging, and add support for a bunch of new HPE Networking and HPE StoreEasy products.

If you have already set the “Automatic Update Level” in IRS to “Automatically Download and Install”, you may already have 7.6 successfully deployed to your server.  It’ll be pretty obvious to tell too – if you see the HPE logo shown above on the login page or as the desktop shortcut icon, you are already at version 7.6.

For some reason however, a couple of my IRS 7.5 servers have failed to auto-update to 7.6.  Trying to install the 7.6 update from the Software Tab in IRS by clicking the Start Update also fails.  Normally at this point, I’d simply go to the Software Depot, download 7.6 and manually run the setup – except that 7.6 isn’t available in the Software Depot as the Software Depot download page generates an error message as of this writing (2016.04.02).

So – after some troubleshooting and poking around the log files, I determined you can download the 7.6 package update from the same spot that IRS downloads it:

https://services.isee.hp.com/SWM/packages/ProdUpgPkg/2016-03-31T154720/ProdUpgPkg+7.6.0.27.zip

Unzip this archive to C:\TEMP and then from a command prompt run:

msiexec /i "C:\TEMP\ProdUpgPkg+7.6.0.27\lib\hprs7kit.msi" /lv "%HP_RS_LOG%\hprs_7.6.0_install.log"

Now – if your servers were like those same servers I have, this will fail too.  Taking a look at “%HP_RS_LOG%\hprs_7.6.0_install.log“, you’ll find that pg_dumpall.exe couldn’t connect to the database as the connection was refused.  This results in database.sql being missing, which causes the install to puke with an error code of 1603.  database.sql is the Postgres database dump of your production IRS database that the installer attempts to make.  Now just above the 1603 error in “%HP_RS_LOG%\hprs_7.6.0_install.log“, you’ll find the actual command line for pg_dumpall.exe, which should be (depending on the vintage of your original IRS install) either:

"C:\Program Files\HP\RS\postgresql_9_win32\bin\pg_dumpall.exe" --host=localhost --port=7950 --username=postgres --file="C:\ProgramData\HP\RS\DATA\PERSISTENCE\UPGRADE\database.sql"
-- or --
"C:\Program Files (x86)\HP\RS\postgresql_9_win32\bin\pg_dumpall.exe" --host=localhost --port=7950 --username=postgres --file="C:\ProgramData\HP\RS\DATA\PERSISTENCE\UPGRADE\database.sql"

Manually running the appropriate version command line from above will result in you being prompted for the postgres user password 6 times.  Unfortunately, this password is undocumented, but by doing some detective work (I won’t be sharing how I found what it was), I’ve determined it to be “edit – removed 2016.04.05 as per a request from HPE“.  So enter this password when prompted each of those 6 times, and you’ll find C:\ProgramData\HP\RS\DATA\PERSISTENCE\UPGRADE\database.sql is created.  Now you can go back and run the installer again from the command prompt:

msiexec /i "C:\TEMP\ProdUpgPkg+7.6.0.27\lib\hprs7kit.msi" /lv "%HP_RS_LOG%\hprs_7.6.0_install.log"

Your upgrade should now complete successfully, and all that is left is to log into IRS, go to the Software Tab and check for updates, and install any remaining updates.

As always – Use any tips, tricks, or scripts I post at your own risk.

 

HOWTO: In-Place Upgrade Windows 7 Pro OEM to Windows 7 Enterprise

Recently we inherited a new client with some very legacy desktops running an OEM version of Windows 7 Pro.  The bad news for us though was there were a few legacy applications installed on those old desktops that we just could not get to reinstall on the new Windows 7 Enterprise edition machines the client purchased from us, so we decided to just P2V the old desktops into the customer’s VMware cluster, and provide the end users RDP access to their old desktop until we could deploy a validated solution to replace those nontransferable legacy apps.  This worked fine for some of the end users, but some of the other end users needed multi-monitor support.  Windows 7 Pro does not support multi-monitors as a RDP host, but Windows 7 Enterprise certainly does.

The problem is though, there is no supported method by Microsoft to in-place migrate from Windows 7 Pro to Windows 7 Enterprise edition.  That said, there is a method (obviously unsupported by MS) to accomplish this without losing your data if you are brave enough to go down that road.

Before proceeding, a full machine backup is recommended (use something like Veeam Endpoint Backup to create a backup, or if it is a VM, take a snapshot).  Download and extract the most recent Volume Licensing Windows 7 Enterprise iso to the C: drive of the machine. (SW_DVD5_SA_Win_Ent_7w_SP1_64BIT_English_-2_MLF_X17-58882.ISO is the most current Windows 7 Enterprise ISO as of 2016.03.28).  And if the machine utilizes disk encryption, decrypt the volumes before continuing.

After the above items are attended to, open an Administrative Command and run:

REG ADD "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /f /v "EditionID" /t REG_SZ /d "Enterprise"
REG ADD "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /f /v "ProductName" /t REG_SZ /d "Windows 7 Enterprise"

Next (do not reboot after you make the above registry edits), from the extracted ISO run setup.exe, and when prompted for the type of installation you wish to perform, select the option to Upgrade.

2016.03.28 - 12.13.19 - SNAGIT -  0000

If nothing is blocking the upgrade (from a compatibility standpoint), then setup will begin an in-place upgrade and reboot the machine several times.  Once the upgrade has completed, you should find yourself with a correctly installed Windows 7 Enterprise Edition instance.

The next step is to activate Windows by installing a Windows 7 Enterprise MAK license key and activating it.  To do so, open an Administrative command prompt and run:

cscript c:\windows\system32\slmgr.vbs /ipk XXXXX-XXXXX-XXXXX-XXXXX-XXXXX
cscript c:\windows\system32\slmgr.vbs /ato

By default the in-place upgrade process “breaks” .NET 4.x, in that Windows does not recognize it is installed, and as a result, any attempt to update, reinstall, or patch any of the .NET 4.x framework will fail.  To fix this, download the current .NET Framework cleanup tool from:

https://blogs.msdn.microsoft.com/astebner/2008/08/28/net-framework-cleanup-tool-users-guide/

Extract the cleanup tool and run it.  In the version drop-down box, select the latest version (4.6.1 as of 2016.03.28), and select “Cleanup Now”

2016.03.28 - 12.23.06 - SNAGIT -  0001

When the cleanup finishes, run cleanup again, and select version 4.6.  Then repeat for 4.5.2, 4.5.1, 4.5, and finally 4.

Next, download and install the prerequisites for Internet Explorer 11 from https://support.microsoft.com/en-ca/kb/2847882.  You can safely postpone rebooting until you get all the prerequisite installed.

These prerequisite include:

http://download.microsoft.com/download/6/C/A/6CA15546-A46C-4333-B405-AB18785ABB66/Windows6.1-KB2729094-v2-x64.msu
http://download.microsoft.com/download/9/F/E/9FE868F6-A0E1-4F46-96E5-87D7B6573356/Windows6.1-KB2731771-x64.msu
http://download.microsoft.com/download/F/1/0/F106E158-89A1-41E3-A9B5-32FEB2A99A0B/Windows6.1-KB2533623-x64.msu
http://download.microsoft.com/download/1/4/9/14936FE9-4D16-4019-A093-5E00182609EB/Windows6.1-KB2670838-x64.msu
http://download.microsoft.com/download/1/8/F/18F9AE2C-4A10-417A-8408-C205420C22C3/Windows6.1-KB2786081-x64.msu
http://download.microsoft.com/download/5/A/5/5A548BFE-ADC5-414B-B6BD-E1EC27A8DD80/Windows6.1-KB2834140-v2-x64.msu
http://download.microsoft.com/download/9/1/C/91CC3B0D-F58B-4B36-941D-D810A8FF6805/Windows6.1-KB2639308-x64.msu
http://download.microsoft.com/download/4/1/3/41321D2E-2D08-4699-A635-D9828AADB177/Windows6.1-KB2888049-x64.msu
http://download.microsoft.com/download/6/1/4/6141BFD5-40FD-4148-A3C9-E355338A9AC8/Windows6.1-KB2882822-x64.msu

After rebooting, download and install Internet Explorer 11.

http://download.microsoft.com/download/7/1/7/7179A150-F2D2-4502-9D70-4B59EA148EAA/IE11-Windows6.1-x64-en-us.exe

Finally you should now be able run Windows Updates and re-install all 217 plus post SP1 Windows 7 updates.

And if you took a VM snapshot before beginning – don’t forget to clean it up now!