D-IPL from direct attached FC tape, no HMC

This would be a great help for our remotely managed distribution center small IBM i systems. We use LAN console, not enough justification for physical or vHMC cost. We have several SST profiles that our team uses. Due to having to D-IPL from a USB device (DVD/USB/RDX) we don't get those restored. According to IBM support there is no way to restore the SST profiles after the full restore/migration is done using Appendix D of the backup & recovery guide.

Thought I would point out the D-IPL rules from IBM:

https://www.ibm.com/support/pages/node/1108539

to support my dissatisfaction with the cost of vHMC on tiny single-LPAR systems which can use only a small fraction of the function provided by the full blown vHMC product, simply for the privilege to do D-IPL from tape

Alternate restart (D-IPL) device rules

If all the following conditions are met, you can do an alternate IPL from a fibre or SAS attached tape drive:

• The system is POWER6, POWER7, POWER8, POWER9, or POWER10 hardware.

• The system is HMC-managed.

• The system is running IBM i 6.1.0 or higher of the IBM i operating system.

• The tape drive is attached to an IOP-less IOA

Let me clarify a few points regarding this high priority and mission critical request (for us).

We are an IBM i software vendor in a small business niche market who sells our application preloaded and bundled with IBM Power servers. These people buy our "bundle" at a price much higher than the usual PC stuff they are used to because of application functionality and the stability of the IBM i on Power vs a PC-Windows based solution (which we also offer BTW, but without the bundled server). The application is mission critical, but there is a "price line in the sand", and it is a huge challenge for us to stay south of that line.

Because these small companies want to focus on their business instead of operating a Power/IBM i, which they know nothing about, not only do we sell them our software bundled with an IBM i server (plus a PC for a console), we also fully manage it for them (which adds to the cost of ownership).

The managed services include running their backups and remotely performing a D-IPL when needed, such as OS upgrades, bare metal re-installs because of a storage failure, or slipping the LIC and/or the OS (recent case was a damaged QTMHHTTP user profile as a result of a power glitch).

We monitor these systems 24x7 so that we can recover overnight to ensure system availability during business hours. So please understand these systems are STANDALONE - there are no LPARS on these machines. They are installed on the customer's premises all over the country. We cannot just quickly drive over and plug in a USB stick or load a DVD for a D-IPL. We can however ship the system with a USB stick premounted, and hope it it is still there and working and have the correct level of LIC when we need it 7 years later. I would not bet my career on that...

Due to the cost of Tape Media Library/VTL solutions and the already steep price for Power/IBM i + a console PC, we currently sell the system with a $300 USB RDX backup device (for which D-IPL support was added only on Power8 and later, BTW). RDX is very reliable and reasonably fast to perform backups, however:

a) we cannot fully automate backups because it requires a human to swap cartridges everyday, and that is easier said than done with these small companies, and

b) a bare metal recovery from RDX takes anywhere from 16-24 hours, so the RTO is off the charts unacceptable.

c) Power7 and older could not D-IPL from RDX (thankfully most customers are now on Power8 and later).

As the service provider we are held responsible for performing the backups and for ensuring system availability. With non-automated backups requiring end user intervention (we cannot just drive over and go and whip them into reliably swapping RDX cartridges), and a terrible RTO on RDX, we are in a no-win situation.

We discovered an open source VTL solution that emulates IBM TS3100 and can run on the "console PC" we already sell with the system. That opens up a an affordable (just pop in a few extra hard drives) solution that can replace the RDX and will enable fully automated backups without human intervention. We have spent 6+ months developing and testing this solution and it is solid and fast. Bare metal recovery is down to 4 hours.

As for the suggestion to use SAS, per https://www.ibm.com/support/pages/ibm-i-removable-media-welcome-page, "SAS tape library devices are not supported..." (besides, plugging in a SAS adapter spins the CPU fans at full speed and a Boeing jet engine is not welcome in an office environment). SAS is not an option...

Problem-1: FC HBA cost an arm and a leg. Solution: e-Bay. We bought 120 x #5735 for less than $100 a piece.

Problem-2: Direct attached FC (no expensive SAN switch required). Certain older FC HBA adapters do support direct attach. (HOWEVER, with the recent Power 10 announcement there is no longer any support for direct attach - see https://www.ibm.com/support/pages/node/6487771)

Problem 3: D-IPL support. Apparently the #5735 is the only FC HBA that supports D-IPL (and direct attach). Again, from what I can tell, the Power10 snuffed that one too.

Problem 4: Requirement for an HMC. Now let me be clear: We love HMC and would be delighted to have one on each customer system (remote power on and many other nice remote management features). However, at $3000 a pop for a standalone system (these customers do NOT have LPAR's), and the requirement to a buy it again each time we upgrade the customer to a new Power-generation, it is ridiclous and silly and I will keep saying that a million times until IBM comes up with a sensibly priced vHMS solution for these small customers, and include free version upgrades in the vHMC maintenance we pay. See also https://ibm-power-systems.ideas.ibm.com/ideas/IBMI-I-2569

Please understand these are independant companies, each with their own brand of firewall and rules. Our access to their networks is very limited. We access the "console PC" via TeamViewer or Log-me-in, therefore the suggestion of a few central HMC's at our location is lovely but not practical, unfortunately. Each of our customers will require a vHMC on their local "console PC".

Problem 5: Power 10. Our upcoming customer box swaps (we have the last few Power7's to migrate in 2022) cannot go to Power10 and will have to be Power9, because the carefully crafted and affordable VTL solution just became economically non-viable with Power10. It now will require an expensive SAN switch and a vHMC and a much more expensive FC HBA than the #5735. So instead of bringing much needed improvements, Power10 not only made it worse, it made it impossible to provide low cost automated hands-free backup and recovery for the small user! So what the heck do we do now?

Finally, just to put things in perspective, as the service provider we require our customers to maintain IBM hardware and software support. Therefore, when IBM announces end of hardware support for an older model, we basically force the customers on those models to upgrade to the latest Power generation. A box swap is therefore required every 5-8 years, resulting in a price tag of nearly $40K ($5K of which is an IBM i license transfer fee, what a rip off), which is enormous for these guys. Now add another $3k for a new vHMC version (despite having paid vHMC maintenance), which is a requirement to support the new Power model. Also add the enormous additonal cost to migrate the VTL solution from Power8 to Power10 (I fear this alone will add an additonal 5-digit price bump for no additional Save-Restore benefit over Power8 or Power9).

IBM is not making it easy for us, and for the small users, to stay on IBM i.

Having done some preliminary investigation, IBM agrees that it would be desirable to provide a solution for doing a D-mode IPL from direct-attached tape without the need for an HMC. It is not trivial to provide such a solution, but IBM will use this request as input to planning. No commitment is made or implied. This request will be updated in the future if IBM implements it. IBM will use votes and comments from others in the community to help prioritize this request.

Thank you for submitting this request. D-IPL scenarios are complex, so it will take some time to investigate further, but we'll investigate to see if there is a way to improve current functionality. Note that no commitment is made or implied. Meanwhile, placing this RFE into Under Consideration to let you know we are indeed looking into it.

Due to processing by IBM, this request was reassigned to have the following updated attributes:
Brand - Servers and Systems Software
Product family - Power Systems
Product - IBM i
Component - Hardware
Operating system - IBM i
Source - None

For recording keeping, the previous attributes were:
Brand - Servers and Systems Software
Product family - Power Systems
Product - IBM i
Component - Save, Restore
Operating system - IBM i
Source - None

The CEAC has reviewed this requirement and recommends that IBM view this as a MEDIUM priority requirement that should be addressed.

Background: The COMMON Europe Advisory Council (CEAC) members have a broad range of experience in working with small and medium-sized IBM i customers. CEAC has a crucial role in working with IBM i development to help assess the value and impact of individual RFEs on the broader IBM i community and has therefore reviewed your RFE.

To find out how CEAC help to shape the future of IBM i, see CEAC @ ibm.biz/BdYSYj and the article "The Five Hottest IBM i RFEs Of The Quarter" at ibm.biz/BdYSZT

Therese Eaton – CEAC Program Manager, IBM

The CAAC has reviewed this requirement and recommends that IBM view this as a high priority requirement that is important to be addressed. The part that is High is the ability to D-mode IPL from FC without an HMC.

Background: The COMMON Americas Advisory Council (CAAC) members have a broad range of experience in working with small and medium-sized IBM i customers. CAAC has a key role in working with IBM i development to help assess the value and impact of individual RFEs on the broader IBM i community, and has therefore reviewed your RFE.

For more information about CAAC, see www.common.org/caac

Nancy Uthke-Schmucki - CAAC Program Manager

For what it's worth and I suspect you won't think it's worth much but here is my opinion:

If you want Lights out management of a system, you really do need a HMC, not just for the recovery but for day to operations.

If you are managing 120 systems, why don't you provide the HMC license yourself, you could have all 120 managed servers on just a couple of HMC's, there is no need for you to put the HMC on each site, you just need network access from you HMC to the ASMI ports. The client can still have an alternate console via Ethernet Ops console if that is desirable.

I'm not sure what recovery scenario you are pitching, as you can delete all ASP's from a LPAR that is managed from today Ops Console and perform a full reload without a D mode IPL.

It is only when you want to re-initialise the system's LIC that you need the D mode IPL. And if you really want to achieve what you want, then you can do it today, simply by preparing a USB based memory stick with a boot image of the LIC on it. You can D mode from that and then use you FC tape interface for as the alternate installation device.

One thing that I don't understand is how your clients can afford FC adapters but not vHMC's, many FC adapters cost more that a vHMC.... Could you not give them a SAS interface and spend the saving on a vHMC license? ;-)

PS
If you want a bit of free advice, when you are asking someone to make an enhancement for you, they are much more likely to do it, if you don't call them "ridiculous" and "silly"

Cheers Brad
Steve Bradshaw
IBM Champion, Member of CEAC, TD of i.UK.co.uk
and by day a Friendly Techie Bloke at RowtonIT.com