3Par

What you need to know about the new HPE Hybrid IT Master ASE Certification exam

@DeanColpittsLeave a comment

As I am sure those of you who are heavily involved in architecting Hewlett Packard Enterprise’s infrastructure solutions consisting of servers, storage and networking already know, there was a new HPE Master level certification announced earlier this year.  This new certification is the HPE Hybrid IT Master ASE, and it is going to be the pinnacle of all HPE certifications.  Many of us that hold Master ASEs in Servers, Storage, and Networking will naturally be looking to obtain this Master ASE certification as well.  In some cases, Partner Ready requirements will drive your need to obtain this certification, but I also know that for many of my peers, it’s a matter of pride and desire to achieve this certification.  However, it really doesn’t matter the reason that drives you to achieve it, I am writing this article to tell you that achieving this new certification isn’t going to be a walk in the park.  HPE opted to take a different path to certification and the traditional testing methods we all know, have tested with before, and are comfortable with have been changed up some for this certification.

By now you are asking yourself how does Dean know about this?  Myself, along with several of my peers from around the globe (many of whom you would likely know too) were honored to be invited join the design team for this certification (and some of related electives for the certification).  When this certification goes live, it will have been a 15+ month journey for some of us, beginning in August 2018.  That journey took us from the initial blueprint of how we wanted to test, to the content of the beta courseware (which was just finished last month), to the certification launch on November 1, 2019.  There are hundreds and hundreds of hours involved amongst us in the design of this certification, the courseware, and of course creating the certification exam its self.  Along the way, there were many phone calls, Skype meetings, face to face meetings at various HPE facilities, and countless hours of reading (and then revising) the alpha and beta courseware material that makes up both the Hybrid IT ASE and HPE Hybrid IT Master ASE courses and exams.  In mid-July (2019) many of us from around the globe gathered in a meeting room at HPE’s campus in Roseville, California to work on the exam creation.

The first thing you’ll notice different is the exam number.  Today, we normally all take proctored HPE0-### exams for our certifications.  The HPE Hybrid IT Master ASE certification will be an HPE1-### series exam, and will not be delivered by Pearson VUE but rather it will be delivered by PSI.  While PSI does have some testing centers, the HPE Hybrid IT Master ASE exam will be an online proctored exam that you will be expected to take at home or at your office – similar to the online proctored HPE0-### exams that are already offered by Pearson VUE.

The second difference you will notice is the length of the exam – you will be given 4 hours to complete it, not the typical 90 or 120 minutes you are used to with the HPE0-### exams (yes – washroom breaks will be allowed).

The third thing you will notice different is both the exam price and the retake policy.  The price of the exam will be between $695 and $895 USD depending on your country of residence, which is more than double the price of today’s HPE0-### exams.  The retake policy is also different.   With HPE0-### exams, you can immediately retake the exam once if you fail it (as long as you have not failed twice in 14 days).  With the new HPE1 exam, there will be an automatic 14-day waiting period after each failure before you can rebook for another attempt.

The fourth thing you will notice is the composition of the HPE Hybrid IT Master ASE exam – it will be broken into 3 distinct sections.  Questions and answers (similar to today’s exams), a research portion, and a hands on portion (more details on all three of these sections is below).  However, for every single item, once you click submit on the answer to the item, there is no going backwards to review or change your answer.

Part one of the exam will consist of a series of Discrete Option Multiple Choice (DOMC) questions.  For those of you that have not seen a DOMC exam before, basically you get asked a question, and are presented with a single answer on the screen at a time – to which you either select YES or NO if the answer is correct for the question.  Each question may have one or more answers that get presented to the test taker (but still only one answer at a time will appear on the screen).  I’ll admit I was very skeptical and concerned when the decision was made to utilize DOMC, but having worked with it for a while now as part of this process, I’m very comfortable with it and I am no longer concerned it will affect your chances of passing or failing.

Part two of the exam will probably start to take some of you out of your comfort zone.  You’ll be given a series of scenarios that you will need to answer questions about.  Some scenarios may build on previous scenarios you were given as well.  You’ll RDP a remote environment, and be required to observe many items in that environment to answer questions about accurately building a solution that properly integrates with that existing environment.  Nothing is off the table here from Synergy frames to storage systems and network switches.  Almost all the Hybrid IT portfolio and their respective management GUIs or CLIs are present here – you’ll need to know where to look to determine if the answer presented to you (via DOMC) is correct.  This is no different from what you’d need to do if you were designing an upgrade for one of your customers.  A simple example is “Your customer wants to do this with their existing environment, do you need to add this particular item to your solution to accomplish this? YES or NO”.

If part two got you out of your comfort zone, then part three is going to really take you far out of your comfort zone…  In part two, you are simply reviewing the exam’s hardware infrastructure and environment, but in part three, you are actually modifying the environment – with very real hardware that you are connected to.  Think of it as having to perform a demo of a feature or something to one of your customers using their existing equipment.

You know all those hands on labs offered at various HPE conferences that you may have attended in the past, but you’ve skipped to spend extra time at the bar in the evenings?  Well those HOL experiences will be very handy here, as it’s very much hands on with the management tools (both GUI and CLI).  Everything from configuring, upgrading, or fixing connectivity issues with Synergy, 3Par, Windows, vCenter, and switches (of all types) is covered here – and you may need to use multiple tools from across the portfolio to accomplish your tasks.  You may use either the GUI or CLI to accomplish your task (or maybe both), but the task must be 100% correct and completed when you hit the submit button.

You will be provided all the appropriate manuals, CLI guides, and documentation you require to complete the tasks – they will available on the server you will be RDPing into.  So it’s opened book so to speak – you’ll have these resources, but only these resources (you won’t be able to search the internet for walkthroughs!).  However, if you have to utilize the provided material to look up how to complete every single little step, you’ll quickly run out of time – the documentation is there to provide you a guide, not tell you how to perform (i.e. for the first time in your life) whatever action it is you need to do.

A word of warning though – as this is real hardware, running in a real datacenter, it is possible for you to completely break the testing environment, which will prevent you from completing your assigned task, possibly resulting a score of zero for the task.  In the real world, if you mess up and accidently destroy or delete something in your customer’s running environment, you’ll have failed in the customer’s eyes.  This is no different – if you break the testing environment here (i.e. maybe you accidentally deleted a volume instead of extending a volume) and are unable to complete the assigned task because of it, then you’ll fail the question.

HPE says this is the first time anyone in the IT certification industry has used real hardware and an automated scoring system in real-time to verify that what you have done is correct.  Spelling counts.  Exactly correct numbers count (i.e. 100MB vs. 1000MB).  If you are asked in a scenario to name something “bigwheel” and you name it “big wheel” with a space (or you typo it as “bigwhel”), then that answer will be marked wrong (although we are told the scoring won’t look at the case sensitivity of the answer, just the spelling, spacing, etc.).  So just like in real life – spelling errors and wrong numbers will result in broken configs, or in this case a wrong answer.  This is completely automated scoring (don’t worry – it’s been fully vetted by your peers already) – so when you hit that final submit button (and I do believe if memory serves me correctly that you’ll be warned that your answer / task is about to be scored if you hit submit), the testing software instantly runs a series of scripts that interrogates everything that makes up the exam’s hardware environment and looks at the relevant output to determine if you’ve correctly accomplished your assigned tasks.  So you’ll know in just a few seconds after hitting that very final submit button if you are the world’s newest HPE Hybrid IT Master ASE or not!

The HPE Hybrid IT Master ASE certification exam is not going to be for the faint of heart.  This certification is going to require you to have several years of real world experience and knowledge in HPE compute, storage, and networking.  And if you think you are going to be able to rely on a brain dump to pass, think again – DOMC, the scenarios on real hardware, the exam cost, and the retake policy (along with some other things I can’t discuss) are going to put a serious crimp on both the quality and quantity of brain dumps that will be available.

So what are my tips to you for achieving this certification?

  • Do take the course.  Yes it is expensive and time consuming, but it will cover (including hands on labs) the concepts and knowledge you must have (aside from the real world experience you should already have) to pass the certification exam.
  • Do not wait to take the exam once you have taken the course – take the exam while the course and hands on labs are fresh in your mind.
  • Be prepared to wait for an exam slot. I think initially it will be hard to schedule an exam due to demand and the limited number of testing slots available per day (given that the exam requires a complete set of real hardware that must be flattened and reset after each exam).
  • Do not wake up one morning and decide to take this exam in the afternoon “cold” without properly preparing.  Many of us do this today at various events we attend (i.e. Aspire, TSS, Discover), and it’s not going to result in an exam pass here.  I know of maybe a handful of my peers in the world that maybe could do that without any preparation and have a reasonable chance of passing.
  • Do read, re-read, and then re-read every single word of every single question on the exam – some of the questions and scenarios are very long with lots of information, and it’s easy to skip over key details, words, or numbers that you will need to accurately answer the question or complete the scenario assignments.
  • Do not be intimidated by the DOMC format – it’s really not as bad as you may initially fear.
  • Do take the practice DOMC exam so you have an idea of what to expect on the real exam. You can find a HPE DOMC practice exam (with examples of ASE level server/storage/networking items) at the following link:  https://sei.caveon.com/launchpad?exam=try-domc-for-hpe

For those of you planning to try to obtain this certification, before you register for the course, I’d suggest you chat with your regional Partner Enablement Manager to see if there are any promotions running for the course and exam (wink, wink, you may find a pleasant surprise).

I would like to wrap up by offering you the best of luck in obtaining the HPE Hybrid IT Master ASE certification and to remind you:

You will truly need to be a Master of HPE Hybrid IT to become a HPE Hybrid IT Master ASE!

 

HOWTO: Install QLogic QConvergedConsole beside HPE 3Par SSMC and Veeam Backup & Recovery

@DeanColpittsLeave a comment

Ok – I’ll admit it – I’m something of a vendor snob…  And my vendor of choice when it comes to Ethernet and fibre channel host connectivity is QLogic and HPE’s OEM products made by QLogic.  You just can’t beat the price or performance of the offerings, and the support that QLogic’s HPE OEM team gives you – they are second to none (a huge shout out to @ToddOwens_QLGC & Jim Burton – if you guys are reading this, thanks for all the amazing support over the years!).

One of the interesting things about QLogic is their branded applications generally work hand in hand with the OEM products they offer to various system manufacturers such as HPE, Dell, and Lenovo.  While I was attending a storage conference last week, I sat in on a presentation Jim and Todd were hosting.  During the presentation the talk turned to QLogic’s comprehensive adapter management tools, including the Web-based QCC (QConvergedConsole), which is supported on Windows, Linux, and Solaris.  QCC allows you to modify and configure your adapters (Ethernet, iSCSI, FCoE, and FC), upgrade the flash on them, perform FC ping and traceroute, and to view reports, statistics, and diagnostics of all the QLogic devices in your equipment – either locally or remote.

Given that QLogic devices are generally so bullet proof, and that the HPE Support Pack for Proliant takes care of my firmware updates, I rarely have a need to install and use QCC.  But today was a little different – I had a VMware host that suffered a Purple Screen of Death overnight, and while I was in the ILO power cycling it and looking for a reason for PSOD, I noticed that ILO was complaining that the 534FLR-SFP+ adapter was degraded because it was in FCoE mode and not connected (we don’t use FCoE).  Since I didn’t want to waste any more time playing around with the host before I brought it back online, I decided that I would load QCC on my management server at the site and see if I could disable FCoE mode remotely.

I never did find a way to disable the FCoE function via QCC – I only spent 3 minutes looking at it, so there may well be a way if I actually RTFM (that isn’t my style though), but this post isn’t about that.  This post is all about getting QCC to co-exist (temporarily anyways) on a server that already has HPE’s 3Par SSMC and / or Veeam Backup & Recovery installed on it.  QCC has been around a long time – longer that both SSMC and VBR, and as such has a few port conflicts that the guys at HPE and Veeam never took into consideration.  As a result, you can’t just fire up the QCC installer and expect it co-exist and run 7/24 right out of the box along side SSMC and VBR.

Once you have the QCC installer downloaded and extracted, there are a few things we need to do before firing up the installer.

First, lets check to make sure TCP ports 8080, 8443, and 111 are not in use.  We can accomplish this by opening an elevated command prompt and running:   netstat -ano | find “0.0.0.0:####”

2017-01-30-15-04-41-snagit-0067

In the example above, you can see that two of the three ports are in use.  Port 8443 is used by the application that has a PID of 38692, while port 111 is used by the application that has a PID of 30000.  Using Task Manager, or better yet my favorite tool for the job – Process Explorer, we can easily determine the applications that are hogging these ports if we enable the PID and Path columns and then sort of the PID.

2017-01-30-15-08-27-snagit-0068

So to get started, we need to stop (temporarily) SSMC and VBR’s vPower NFS service.

2017-01-30-13-56-34-snagit-0040

2017-01-30-13-57-00-snagit-0041

Now that we have stopped these two services, lets double check to make sure TCP ports 8080, 8443, and 111 are no longer in use.

2017-01-30-15-13-24-snagit-0069

So with all three ports now free and no longer in use, we can launch the QCC installer as Administrator (note – all screen snapshots are based on Windows 2012 R2 with QCC v5.4.0.41).  Click next a couple of times until you get to the “Please enter desired port number”.  This defaults to TCP 8080, which as we checked already above, is free to use, so go ahead and click Install.

2017-01-30-13-58-32-snagit-0046

Eventually the installer will prompt if you wish to restrict access to localhost.  No one else at my sites require access to QCC, so I’m ok with restricting access – I clicked yes (note it defaults to no, so if you just hit enter you answered no…)

2017-01-30-13-59-39-snagit-0048

Eventually, you’ll be prompted if you wish to enable security login.

2017-01-30-14-00-02-snagit-0050

Since this application is only going to be enabled temporarily when I actually need it on the management server on the management VLAN, and because I am restricting access to the localhost only, I left the checkbox cleared.  That said, you may wish enable security – and if you do, make sure you make a note of the credentials you set!  The default login id credentials if you didn’t change them is “QCC” with a password of “config”.  Click Next to continue.

Now you are prompted if you wish to enable SSL.  That is likely a good idea, even if you are restricting it to the localhost – so click yes.  This will automatically set the Tomcat7 engine to use TCP 8443 and you can not change this from the installer.

2017-01-30-14-00-21-snagit-0051

Finally you will be presented with the Done button.

2017-01-30-14-02-22-snagit-0052

Take note of the URL as you will need it shortly…  https://localhost:8443/QConvergeConsole/ or http://localhost:8080/QConvergeConsole/

Now we can go ahead and install the necessary management agents.  In my case I am going to install all of the management agents.

2017-01-30-14-02-55-snagit-0054

After we click Next, you’ll notice that the installer is installing the ONCPortmap service.  This runs on TCP 111.  If TCP 111 is already in use, the installer will hang, and hang, and hang…  This is why we stopped the Veeam vPower NFS service earlier.

2017-01-30-14-03-14-snagit-0055

Eventually the management agent will complete the install process.  When we install the next management agent, you’ll notice a warning about the ONCPortmap service – this is good!  It means the ONCPortmap installed and started successfully.

2017-01-30-14-04-05-snagit-0058

After we have all the management agents installed that we want or require, we can go back in the command prompt and check our port status again.

2017-01-30-14-08-10-snagit-0059

Now you can see that all three ports are in use – which means QCC is likely ready to go.  Sort of…  As I mentioned previously, because the ports used in QCC conflict with SSMC and Veeam vPower NFS, we can’t just leave things alone and expect all three apps to work in the future after a reboot.  In my environment SSMC and Veeam are more important than QCC, and I always want them to be started after a server reboot.  So we need to set the follow services to be manual start instead of automatic (which they are by default) so they don’t prevent SSMC or Veeam from starting.

  • ONC/RPC Portmapper
  • QLManagementAgentJava
  • QLogic Management Suite FastLinQ
  • QLogic Management Suite Java iQAgent
  • QLogicManagementSuitenQLRemote
  • Tomcat7

Once we have changed the startup type of these services to manual, then lets login using the URL we were shown above.

2017-01-30-14-08-58-snagit-0060

Now – in the Host Selection dialog box, type in localhost and hit the connect button.  You should be able to safely ignore any errors you may see.

2017-01-30-14-09-18-snagit-0061

Finally – the console is opened!  Lets make a simple cosmetic change to see if it works (so something that does not affects the performance or anything of the adapters).  Highlight one of the ports of one of your adapters (in my example below, Port 0 of the HP 533FLR-T) and click on the MBA Boot Cfg tab in the right hand pane.  In the Hide Setup Prompt drop-down box, pick the opposite of whatever is there (it is probably already disabled, so select enabled), then click the Apply button.

2017-01-30-14-11-55-snagit-0063

You’ll be prompted for a password.  This password, assuming you made no changes to the default setup will be “config”.  If you aren’t sure if this is correct, clear the checkbox that says save password.  If you leave it checked, and the password you put in is wrong, then you will need to log out of QCC and back in to be able to try a different password.

2017-01-30-14-12-14-snagit-0064

If you had the correct password, you’ll see a green banner advising you of a successful update!

2017-01-30-14-12-31-snagit-0065

Now all that is left is to make the changes you actually set out to do!  Of course, once you are finished, you have two choices – reboot the server to apply the changes and have SSMC and VBR startup on reboot, or ignore the reboot, manually stop all the QCC services (see the list above) and manually start the SSMC and VBR services.

Now that you have QCC installed, if you need to access it in the future, you can just stop SSMC and VBR, then start the necessary QCC services.  While it isn’t a perfect solution, it will allow QCC to coexist along side both SSMC and Veeam’s vPower NFS service.

As always – Use any tips, tricks, or scripts I post at your own risk.

HOWTO: Replace a failed 3Par drive

@DeanColpittsLeave a comment

HPE 3Pars are great arrays, but just like any other storage system, they do occasionally end up suffering a failed hard drive.  Replacing a failed 3Par drive isn’t quite the same as replacing a failed Proliant Smart Array controller drive – there are a few manual steps that need done to facilitate the replacement process, which I am going to detail below (note – I’m using a StoreServ 7200, based on OS 3.2.1 MU2 as my reference in this post).

First, SSH (via Putty) the 3PAR’s management IP and login as 3paradm (remember the username and password are case-sensitive).

At the 3PAR_SN# cli% prompt, type:    showpd -failed -degraded

This should show you the failed drive and it’s ID (in the example below, the drive hasn’t totally failed, but rather is just degraded due to an internal loop error in the drive, so it needs replaced).

2016.05.24 - 09.15.46 - SNAGIT - 0026

Next, see if servicemag has been issued or is running with:   servicemag status

If servicemag is not running, you will see:   No servicemag operations logged.

Now we want to see if the data has been evacuated off the drive already by running this command:   showpd -space 15   (where 15 is the drive ID that needs replaced).   Using the output shown below, double check there is no data left on the drive. You need to check that all columns other than size and failed are zero.  As you can see from the example , this drive still has data on it (again because the drive in this example is only degraded, not failed – my experience is that typically failed drives have 0, 0, 0, 0 for volume, spare, free, and unavailable, while failed is usually equal to the size).

2016.05.24 - 09.15.54 - SNAGIT - 0027

To evacuate the data, run this command:    servicemag start -pdid 15     and answer yes when prompted if you are sure you want to run it.

2016.05.24 - 09.56.05 - SNAGIT - 0033

To check the status / progress of the servicemag command, run:    servicemag status

2016.05.24 - 09.16.14 - SNAGIT - 0029

As you can see above, 4 chunklets (1GB blocks of disk space) have been moved off the drive so far, with another 107 chunklets (107 GB) to evacuate.  Below is what you will see once the servicemag process has finished.

2016.05.24 - 09.16.23 - SNAGIT - 0030

Before continuing, verify there is no data left on the drive by running:  showpd -space 15

2016.05.24 - 09.16.28 - SNAGIT - 0031

When the HPE field engineer arrives onsite with the replacement disk, you may need to turn on the locate light on the failed drive for him.  To do this, run:      locatecage -t XX cageY ZZ    where TT is time in seconds (i.e. 300), and Y in cageY is the cage number shown above, and ZZ is the magazine number to locate (i.e.  locatecage -t 300 cage0 15 enables the flashing locate light for 5 minutes for the failed drive that is being referenced in this HOWTO).

Once the drive has been replaced, the 3Par **should in the background** run an admitpd automatically for you.  To verify this, run:   showpd -p -mg ZZ -c Y     to see if the new drive is listed (note it will most likely have different drive ID than the dead drive)

When you have verified the new drive has been seen and admitted, you can check the rebuild status with servicemag status.  You can see below the rebuild process, followed by the status message once servicemag as successfully finished.

2016.05.24 - 09.16.52 - SNAGIT - 0032

If you go back to the HP 3PAR Management Console and refresh the console, you should find the fail drive no longer appears (it will stay there appearing as failed even after it has been removed from the cage until the rebuild process is completed, at which point it will go away).

If the HP 3PAR Management Console indicates a firmware update needs performed on the replacement drive, run:   upgradepd ZZ    and answer yes when prompted.  Refresh the HP 3PAR Management Console when the upgrade is complete to check for any other errors.

If no further errors appear, the drive replacement process is completed.  If there are errors, then escalate back to HPE with your original case number.

As always – Use any tips, tricks, or scripts I post at your own risk.

HOWTO: Make a 3Par RJ45 serial console cable for an OpenGear ACM5004 Remote Site Manager

@DeanColpitts2 Comments

On all the 3Par StoreServ 7000/8000 nodes we manage for our clients, we utilize OpenGear ACM5004-F-E Remote Site Manager (basically a serial console switch) so that we can remotely access the serial console on the nodes (in case a node is shutdown to the SMI Whack prompt or something else silly is going on).  The ACM5004 has 4 x RJ45 serial ports on it, which are a Cisco Straight pinout.

Just plugging a standard patch cable into the ACM5004 and the back on the StoreServ node isn’t going to work.  And using the RJ45/DB9 couplers that the 3Par ships with is both a pain in the butt, an eye sore, and bulky.  So I set out to build my own custom RJ45 cable.

On the 3Par end, use pins 2 (orange), pin 3 (blue), and pin 5 (green). Be sure to label this end as the 3Par end!!! I attempt to always put a small tiewrap (which is white in the photo below and hard to see thanks to a blurry photo) around this end of the cable to mark it as the 3Par 3nd.

On the ACM5004 end, use pins 3 (blue), pin 5 (green), and pin 6 (orange). Be sure to label this end as ACM5004!!!

Plug the 3Par end into the node’s serial console port, and the other end into the ACM5004.  After configuring the ACM5004, SSH it and connect the appropriate port on the ACM5004 and you should now see your 3Par serial console.

3Par
3Par End
ACM5004
ACM5004 End