Mobile alerting with tracking & escalation for DataDog
Why Enterprise Alert
Datadog is a monitoring service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform. Persistent notifications and tier escalation within Enterprise Alert ensures that critical alarms are not missed. Issues are taken beyond the dashboard and inbox and delivered via push, SMS and voice.
How it Works
Enterprise Alert establishes a connection via WebHook URL. Using the REST API, event data will trigger alert policies on key parameters so that users receive only the most critical alerts with the most pertinent information. Each team member will receive an alert via their designated notification channel, allowing for the flexibility of user dictate notification methods.
Forwards Datadog alarms via webhook to Enterprise Alert for mobile alerting
Staff can acknowledge and take ownership of critical incidents that occur
Communicate within an alert to address a problem
Alerts are escalated in case of no response
On-call scheduling to alert the right people at the right time
How to Integrate
Sending Datadog Alerts to Enterprise Alert
First, lets set up the WebHook integration within Datadog. From the left menu select Integrations and choose the WebHook integration.
Populate the name and URL (you get that from the next section) as well as the payload to be submitted.
Edit an existing Monitor and on the 5th step add the webhook as a notification channel.
The next time the alert triggers, the information related to the event will be sent to the REST API and Enterprise Alert will evaluate whether this event triggers an existing alert policy.
Create a new Rest API endpoint
In Enterprise Alert click on the System-Event-Sources menu options.
In the Event Source page click on the New Source button at the bottom.
Select REST API Client from the dropdown list, add a name for the new source and save the API Key somewhere for use in step. Click the Save button.
In Enterprise Alert there is an Endpoints section on the home page, copy the REST API URL and paste it in the Datadog WebHook configuration, append the API key from step 3 at the end of the line after the “apiKey=” portion.
Creating alerts in Enterprise Alert for Datadog Alerts
To receive Alerts from the previously created REST Event Source Alert policies must be created. To do that please follow the below steps:
Click on the Alert/Incoming Events button
Find a Rest API alert from Datadog and click on the “Create Policy From Event” button
Give the event a Name, the Event Source will default to Events from Datadog. Click Next.
The Conditions will fill by default with the variable provided by the Datadog event. Set the conditions to meet the needs of your requirements. These can be anything from as simple as the email address to as complex as using specific keywords in the error message and using multiple and/or statements to make the alerts more focused. Click Next.
Choose the users/teams you wish to be alerted and the notification channels you wish to use for the selected teams. Click Next.
Enter a Subject and Body of the alert to send to your users. Enterprise Alert allows the use of dynamic content, this forwards the Datadog information such as Subject, Text(body), and Originator Address to be included in the message from Enterprise Alert. Click Save.
Notifications within the Enterprise Alert mobile app
The Enterprise Alert mobile app shows alerts for your users. Below we have examples of an open alert, alert details, and a closed alert from the Enterprise Alert mobile app.
A centralized alert center can be viewed from the web portal. All alerts, whether active, overdue or closed are visible to EA admins. Ownership and even delegation is as easy as drag-and-dropping the alert to the person/team.