Reduced Router Redundancy

Full redundancy has now been restored with no impact to service observed (not even VPN tunnels).

Reminder. The work to replace the failed storage controller battery module is going ahead as scheduled this evening. The scheduled start time is 2200 and we expect to be finished by 2330.

Whilst this is a fully redundant service network connectivity will be disrupted for around a second during each of the two failover events. Most customer site-to-site VPN tunnels will also drop and re-establish twice during this period, with a less predictable period of interruption but still in the order of seconds. We will monitor all tunnels and intervene where necessary and ensure all are online again at the end of the maintenance period.

Router redundancy has now been restored.

During this work we observed a failed storage controller battery module in the other (live) router. We have spares in stock and will replace this after 2200 on Tuesday 17th March alongside other (unrelated) upgrade work already planned.

Whilst this is a fully redundant service, during the battery replacement most customer site-to-site VPN tunnels will drop for a few seconds when we fail over to the standby router. If we decide to fail back to the original live router then there will be a second drop - also for a few seconds. We will manually check that all tunnels are live following this maintenance.

We will post another update before that works starts, and a final update once complete.

One of our redundant routers is currently suffering an issue and requires a reboot and possible physical intervention. There is currently no impact to services - only reduced redundancy. Due to no current impact we have chosen to schedule work on this issue from 2030 this evening.