The pressure on IT budgets, lack of skilled workforce and the need to deliver IT on time and to guarantee its availability requires an increasing level of automation of processes and practices. Hence, the pace of IT automation is accelerating and promises to reduce risk, costs and to improve operational excellence. Generally, IT automation requires a high level of organizational maturity, standardization of tools and systems and a skilled workforce to implement it. So, it seems a good advice to look for low-hanging fruits, i.e. processes that are already largely standardized and can get automated with ease.
Automation of IT alerting
We believe that the automation of IT alerting combined with an anywhere front-end for IT task automation is such a low-hanging fruit in IT automation. The process of IT alerting, which in many organizations still includes quite a large number of manual steps, can be standardized easily using the right tool, providing great benefits.
1. A cornerstone of the IT alerting process is a targeted delivery of incident alerts to responsible staff. Such responsibility map (IT systems and the team taken care) is usually existing and sometimes even documented in a CMDB. If not, it is implicitly available. With capable mapping technology of a suitable alerting solution it is easy to implement such mapping and thus automate targeted alerting.
2. Another cornerstone of IT alerting is the knowledge of the availability of IT engineers during after business hours (on-call duty schedule). This is commonly managed in an on-call duty sheet. Instead of Excel-based planning, alerting systems like Enterprise Alert provide an integrated on-call duty scheduler. The stored duty records can then be used to automatically by the alerting system to find an available on-call IT engineer to take care of an incident.
3. Another crucial element of IT alerting is the escalation, ideally in real-time upon non-response or non-delivery of alerts. Commonly, this is a manual process in a NOC or SOC, too. However, with the proper communication capabilities like message delivery tracking and 2-way communication to process replies, the foundation for automated escalation procedures is created. All that is needed on top is a stored escalation path, ideally related to the type of the incident or its severity.
4. As IT alerting usually sits on top of IT monitoring, integration with such 3rd party system is essential because incident events need to be retrieved automatically from the source by the alerting system. The level of integration plays an important role as alerts need to stay in synch across all systems (for instance avoiding an ongoing alerting workflow in case of an quick incident remedial). Ideally, integration is both codeless and 2-way.
Once the above listed requirements are comprehensively covered IT alerting can be automated with ease.
Bye, bye manual alerting
Instead of a typical alerting process and setup where
- NOC engineers stare at dashboards
- Wait for red blinking symbols
- Identify the incident type and the responsible team
- Look up the on-call engineers
- Manually call out to this person with a verbal error message (or use a webmask to trigger some sort of semi-automated call)
- Escalate manually if the engineer on-call cannot be reached
- Create an incident ticket
the entire process can run fully automated, unattended, error-free and in real-time!
Main benefits of the automation of the IT alerting process typically are:
- Removal of human latencies (e.g. lookup of on-call duty information) leading to much faster alerting
- Less errors in addressing wrong teams with incident alerts
- Removal of alerting-related workload in the NOC in the moment of critical incidents
- Automated generation of meaningful, actionable alert messages
- Reduced risk of overlooking incident alerts
Derdack’s Enterprise Alert adds another value to the automation of the IT alerting process. It enables a true anywhere incident response and management by providing mobile apps as an IT task automation end-point. From the Enterprise Alert mobile app you can easily trigger IT automation workflows, e.g. in HPE Operations Orchestration, System Center Orchestrator or Azure Automation. Workflows can include troubleshooting as well as neutralization or remediation actions.
Feel free to contact us and to start a conversation how numerous enterprise customers like Daimler fully automated their IT alerting process.