ESXi /bootbank Alarms After Disk Replacement: What This Incident Did and Did Not Prove
- Get link
- X
- Other Apps
Operator-reported incident with documentation-based safety notes. Revised September 12, 2026. The sequence below was supplied by the operator. Exact alarm text, ESXi build, boot-device layout, log extracts, and a measured observation duration were not supplied. The root cause remains unconfirmed.
What was actually reported
- vCenter raised an alarm for another hardware object on one host.
- A failed physical disk was found and replaced.
- Afterward, /bootbank warning emails recurred—sometimes within five minutes, sometimes at roughly hourly intervals.
- The operator and hardware maintenance vendor placed the host in maintenance mode and rebooted it.
- The emails stopped after some additional time. Continued observation found no further problem, and the host is currently in service according to the operator's account.
This is a recovery observation, not proof that the replaced disk held the bootbank, that a RAID fault caused the warning, or that restarting repaired damaged media. We also cannot distinguish a continuing host fault from delayed or repeated notification using email alone.
The diagram summarizes the reported sequence, not a universal repair procedure. Read the reboot safety gate below.
The most useful missing evidence
- Original event versus email delivery: record the event timestamp, last occurrence, alarm state, and notification time. A late email may describe an earlier event.
- Disk role: identify the replaced device and determine whether it belongs to boot media, a data datastore, or another subsystem. Do not infer this from physical proximity or replacement timing.
- Boot access: preserve relevant host logs and the vendor's findings before a restart removes transient evidence.
- Build and topology: record ESXi version, boot layout, controller state, and maintenance changes before matching a knowledge-base article.
Reboot safety gate—not a universal fix
Broadcom documents cases in which connection to bootbank storage is lost and warns that a host may not boot again. Maintenance mode protects workloads during a planned operation; it does not repair the boot device or guarantee restart success.
Before repeating this incident's recovery action, have the platform and hardware owners assess boot-device accessibility and restart risk. Confirm workload evacuation or an approved shutdown plan, spare cluster capacity, out-of-band console access, appropriate configuration recovery material, and a supported recovery/reinstallation path. Preserve evidence and arrange the maintenance window. If safe recovery cannot be assured, escalate before rebooting rather than following the diagram mechanically.
Verification should test the host, not only the mailbox
Compare newly generated host events and current alarm state with email history. Check hardware and storage health, host reconnect behavior, and representative workload operations under the site's normal checks. Select an observation window that covers the earlier recurrence pattern and scheduled activity; this report cannot supply a tested universal duration.
If the warning returns, stop treating the earlier reboot as a proven repair. Preserve the new evidence and decide with the vendor whether the host should be withdrawn from placement again. A rollback here is a safe service/recovery plan, not undoing the disk replacement or repeatedly power-cycling questionable boot media.
What another operator can take from this case
The useful lesson is the distinction between replacing a visibly failed component, restoring service, and proving the root cause. The reported host recovered after maintenance and reboot. Which mechanism caused that recovery was not established. Keep that boundary in the handover so a future incident starts with evidence rather than an assumed “reboot fixes bootbank” rule.
Vendor references
- Broadcom KB 317914: bootbank cannot be found
- Broadcom KB 318611: interrupted bootbank connection and restart risk
Comments
Post a Comment