Did the Oct 21 DynDNS affect your alert notification system?

Businesses who extensively rely on the availability of public Internet services should re-evaluate resilience of their mission-critical systems in case of outages and failures of the public Internet. This applies to systems that represent the operational backbone including incident notification and response systems.

The DDoS attack on DynDNS, one of 14 critical DNS providers, was a black swan event to many Internet-born/based service companies including Amazon, Twitter and many others. But the DDoS attack also heavily impacted a broad range of more critical services, systems and companies. Friday’s event again demonstrated the vulnerability of today’s public Internet. And it is unlikely that the number of attacks on the public Internet is going to decrease. Quite the opposite.

Resilience of mission-critical infrastructure

Operators of mission-critical infrastructure like energy creation, transport, financial and utility companies are aware of the potential threats. They are a big bullseye for malicious activities. Those companies have IT and OT policies in place ensuring the highest standards of availability and security. The goal is simply to protect the backbone of our daily life. One of the major objectives is resilience even in case of large-scale disasters. This would even include the breakdown of critical IT infrastructure or even the loss of an entire datacenter. These companies can maintain their operations without the vulnerable public Internet. They run private networks, private clouds, on-premise software systems and often even remote backup sites that are physically independent.

Though total independence from the public Internet is no option if you use it for your customer relationship (like Amazon) or for delivering virtual goods (like Spotify or Twitter). However, we all should thoroughly look at methods, processes and IT architectures that can mitigate the negative impact.

A few recommendations

From our specific business perspective, we’d for instance recommend to keep monitoring of your IT assets and the ability to communicate with customers or to alert your operational teams in a safe place. Ideally in your private cloud and – above all – along with your other IT assets! A monitoring systems on the public Internet cannot monitor if the very Internet fails. Nor can a notification system alert your engineers if it is hosted on Amazon and uses Internet-based communication providers in case your on premise database fails.

Clearly, it is impossible to entirely disconnect a business from the Internet. But either your core or at least your emergency operations have to be as resilient as possible. Take a serious look at your infrastructure and cloud migration plan. There is no black and white. No public cloud or private cloud only. As always, the truth might be somewhere in between.

Many of our customers including large manufacturers have a clear view on the “core” systems. “Core” systems need to reside within their private clouds on premise. “None-core” systems can be safely migrated to the cloud.

Our incident notification and response solution usually resides at the core, i.e. along with other critical OT& IT assets, often on premise. In manufacturing for instance our alert notification solution is hosted together with the plant control systems on premise.

Further reading

For a highly resilient alert notification and incident response solution that is used in various mission-critical scenarios, please have a look at Enterprise Alert.