Setup hourly HPE Insight Remote Support Service checking

In a previous post, I mentioned we utilize HPE Insight Remote Support (IRS) at all our client sites, and discovered the lovely undocumented “feature” that IRS has, which is a tendency not to start after a Windows server reboot after an IRS update. This great undocumented feature defeats the entire purpose of IRS – monitoring and alerting your HPE hardware. After getting burned by this feature three or four times in a month where customers noticed hardware faults (via amber alert lights on the equipment) before we did since IRS was not running to alert us, I decided it was time to write a script to check IRS hourly and alert us if it wasn’t running.

To configure Windows to send an alert if the HP IRS Service is stopped, create the following two files (file contents are at the end of this post) on the IRS server:

  • check_irs_service_status.cmd – which is the wrapper that will call PowerShell from Task Scheduler
  • check_irs_service_status.ps1 – which is the actual PowerShell script that executes the service status check

Lastly, we need to schedule check_irs_service_status.cmd to run hourly. I’ve set 2 minutes after the hour in the example shown below, but you can adjust as required.

schtasks /create /tn "Hourly IRS Service Check" /tr c:\Windows\check_irs_service_status.cmd /sc minute /mo 60 /st 00:02:00 /rp "*" /ru "%userdomain%\%username%"

By default, the SMTP from address will be the netbios computer name of the IRS server @ the User’s DNS Domain FQDN (i.e. IRS-SERVER@JBGEEK.NET).  The SMTP to address will be support @ the User’s DNS Domain FQDN (i.e. SUPPORT@JBGEEK.NET), and the SMTP server will be mail @ the User’s DNS Domain FQDN (i.e. MAIL.JBGEEK.NET).  You can determine what these will be by checking the system’s environment variables with SET from a command prompt.  You can customize these settings in the “Send-MailMessage” command if necessary.

All that is left to do is to stop the service and test run check_irs_service_status.cmd to verify the Send-MailMessage works properly in your environment.

 

check_irs_service_status.cmd

rem --- begin cut and paste of notepad c:\windows\check_irs_service_status.cmd
@echo off
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy RemoteSigned -noprofile -File C:\Windows\check_irs_service_status.ps1
exit /b
rem --- end cut and paste of c:\windows\check_irs_service_status.cmd ---

 

check_irs_service_status.ps1

###--- begin cut and paste of notepad c:\windows\check_irs_service_status.ps1
### Check_irs_service_status.ps1
### @deancolpitts – http://blog.jbgeek.net
### 2016.01.27
### This script will check the status of the server HPRSMAIN and alert via email if the service is stopped.

$Service = Get-Service -name HPRSMAIN
$Service.Status
if ($Service.Status -eq "Stopped") {
 $CurrentTime = Get-Date
 Send-MailMessage -From "$env:computername@$env:userdnsdomain" -To "support@$env:userdnsdomain" -Subject "$env:computername - HP IRS Service is stopped!!!" -Body "The HP IRS Service is stopped on $env:computername.$env:userdnsdomain at approximately $CurrentTime." -Priority High -DNO onSuccess, onFailure -SmtpServer "mail.$env:userdnsdomain"
}

###--- end cut and paste of notepad c:\windows\check_irs_service_status.ps1

 

HOWTO: Schedule Daily Netscaler VPX Reboots via Powershell

We often utilize Citrix’s NetScaler VPX running on VMware ESXi 5.5 to allow our clients to securely connect to their Citrix infrastructure from outside the firewall.  For the most part – it works well.  Unfortunately though, our experience has taught us that occasionally NSVPX goes all fubar on it’s own after a few days of running and stops processing connection requests once the user logs in.  A simple reboot of the NSVPX VM usually resolves the user’s connectivity issues..

To combat this issue, I wrote a Powershell script that we run as a daily scheduled task on our management server to have vCenter automatically restart the machine once a day.  You could easily modify this script to reboot any VM you want though.

To configure daily VM rebooting, the current VMware PowerCLI client needs to be installed on the machine that will be running the scheduled reboot.  Once the VMware PowerCLI is installed, you need to create 3 files on the management machine:

  1. daily_nsvpx_reboot.cmd – which is the wrapper that will call PowerShell from TaskScheduler (see below in for cut and paste of the file contents)
  2. daily_nsvpx_reboot.ps1 – which is the actual PowerShell script that executes the reboot (see below for cut and paste of the file contents)
  3. daily_nsvpx_reboot.pwd – which is an encrypted file that contains the vCenter user’s password

To create the file daily_nsvpx_reboot.pwd, open PowerShell and run the following command:

read-host -assecurestring "Enter Password" | convertfrom-securestring | out-file c:\windows\daily_nsvpx_reboot.pwd

At the “Enter Password” prompt, enter the password of the user account you will be using that has rights in vCenter or the ESXi host to perform VM restarts.

You may also need to set the PowerShell Execution Policy to support remote signed scripts such as daily_nsvpx_reboot.ps1.  To do this, open PowerShell and run the following command and select Yes when prompted:

Set-ExecutionPolicy RemoteSigned

After creating daily_nsvpx_reboot.cmd and daily_nsvpx_reboot.ps1 (see below for file contents of these two files), edit daily_nsvpx_reboot.ps1 and adjust the variables for $server, $user, and $vm2reboot to fit your environment (these three variables are all defined at the top of the script).

Lastly, you need to schedule daily_nsvpx_reboot.cmd to run daily.  I’ve set 4:15 am local time in the example shown below, but you can adjust as required.  To schedule the task, open an Administrative command prompt and run the following command (adjust domain\username to be the same user account that has rights in vCenter or the ESXi host to perform VM restarts):

schtasks /create /tn "Daily NSVPX Reboot" /tr C:\WINDOWS\DAILY_NSVPX_REBOOT.CMD /sc daily /st 04:15:00 /rp "*" /ru "domain\username"

All that is left do now is test run daily_nsvpx_reboot.cmd and see that it runs and reboots the NSVPX.  If you are monitoring via ProcExp or TaskManager on the management machine, you should note low CPU usage followed by several spikes up to 50% (it is single threaded), and you should be able to see in the NSVPX console via vCenter when it reboots.

And as always – Use any tips, tricks, or scripts I post at your own risk.


daily_nsvpx_reboot.cmd – file contents

rem — begin cut and paste of notepad c:\windows\daily_nsvpx_reboot.cmd
@echo off
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy RemoteSigned -noprofile -File C:\Windows\daily_nsvpx_reboot.ps1
exit /b
rem — end cut and paste of c:\windows\daily_nsvpx_reboot.cmd —

daily_nsvpx_reboot.ps1 – file contents

    ###— begin cut and paste of notepad c:\windows\daily_nsvpx_reboot.ps1

    ### Daily_nsvpx_reboot.ps1
    ### @deancolpitts – http://blog.jbgeek.net
    ### 2015.01.02
    ### This script will attempt to perform a graceful VM restart via the VMware Tools inside the guest.

    ### Variables – please only adjust server, user, and vm2reboot.  Any other variables should not be touched.
    ### Server is the vCenter server or ESXi host’s FQDN, while user is the vCenter user or ESXi user account.
    ### if any smtp variables present, they should be self-explanatory.

    $server = “vcenter.domain.fqdn”
    $user = “vcenter_username”
    $vm2reboot = “nsvpx”

    ### Read the encrypted user password from “c:\windows\daily_nsvpx_reboot.pwd”
    ### Use the following commented out PowerShell command to manually create a new credentials store.
    ### Enter the user’s password when prompted while running the read-host command
    ### read-host -assecurestring “Enter Password” | convertfrom-securestring | out-file c:\windows\daily_nsvpx_reboot.pwd

    $credentialFile = “c:\windows\daily_nsvpx_reboot.pwd”
    $pass = cat $credentialFile | convertto-securestring
    $credentials = new-object -typename System.Management.Automation.PSCredential -argumentlist $user,$pass

    add-pssnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue -WarningAction SilentlyContinue | Out-Null

    if ( $DefaultVIServers.Length -lt 1 )
    {
    Connect-VIServer -Server $server -Protocol https -credential $credentials -WarningAction SilentlyContinue | Out-Null
    }

    Restart-VM -VM $vm2reboot -RunAsync -Confirm:$false

    ###— end cut and paste of c:\windows\daily_nsvpx_reboot.ps1 —