Incidents Overview

An incident represents a problem or an issue that needs to be addressed and resolved. Incidents trigger on services, and a service's escalation policy prompts notifications to on-call responders to remediate the issue.

Incident Statuses

• Triggered: An active service (i.e., someone is on call and the service is not disabled or in maintenance mode) triggers an incident when it receives an event. The incident escalates in accordance with the service's escalation policy . By default, PagerDuty sends notifications when an incident is triggered, but not when it is acknowledged or resolved. Users can create their own notification rules — or use webhooks — to receive notifications when an incident is acknowledged or resolved.
• Acknowledged: An acknowledged incident that is being worked on, but is not yet resolved. The user who acknowledges an incident claims ownership of the issue and halts the escalation process. While an incident is acknowledged, notifications are not sent until the acknowledgement timeout is reached. If the acknowledgement timeout is reached, the incident returns to the triggered status, and the escalation process resumes.
• Resolved: A acknowledged incident has been fixed. Once an incident is resolved, no additional notifications are sent. You can reopen it if further work is needed.

Priority, Urgency and Severity

• Priority is tied to incidents, and it specifies the order in which incidents must be addressed (e.g., P1, Sev-1). Refer to Incident Priority for more information.
• Urgency is tied to incidents and determines how are notified when an incident is assigned to you (high or low). Refer to Notification Urgency for more information.
• Severity is tied to alerts and describes the impact on a specific service or piece of infrastructure (e.g., critical, warning, error). Refer to Event/Alert Severity Levels for more information.

Incident Lifecycle

1. Received through Services

PagerDuty receives events from monitoring systems via integrations. An event creates an alert and an associated incident in PagerDuty.

2. Assignment via Escalation Policies and Schedules

Unlike an alert or a suppressed event, an incident must be assigned to a user. The escalation policy determines to whom an incident is assigned. An escalation policy has one or more levels and can target either a schedule or a user. An incident escalates through the layers of an escalation policy until it reaches someone on call who is notified and assigned the incident. If the user fails to acknowledge the incident before the escalation timeout, the incident escalates to the next escalation level.

Incidents are only created when an escalation policy has an on-call user. In other words, if there is no one to assign an incident to when an event is sent to PagerDuty (due to a coverage gap on a schedule, for example), no incident is created.

3. Notifications via Push, Phone, Slack, Email, or SMS

Each user configures notification rules in their user profile. PagerDuty contacts users according to their notification rules until the incident is acknowledged, resolved, or escalated, either manually or due to escalation timeout.

4. Acknowledging and Resolving

Notifications allow responders to acknowledge that they are working on an incident or that it has been resolved. Depending on a their permissions, users who are not currently assigned to an incident can acknowledge or resolve it on the Incidents dashboard in the web app.

Resolving an incident closes the incident, while acknowledging only halts escalation. If the incident is not resolved before the service's acknowledgement timeout expires, it retriggers and continues to escalate.

Incident Timeline

Each incident has a Timeline tab in the incident details page, showing timestamps of each incident status along with all other actions taken and notifications sent from the incident.

You can filter the incident timeline by specific events and actions, making it easier to track particular types of updates or changes and review patterns throughout the incident.

Trigger/Declare an Incident

There are multiple ways to trigger PagerDuty incidents depending on your use case:

Acknowledge an Incident

There are multiple ways to acknowledge PagerDuty incidents depending on your use case:

Unacknowledge an Incident

If you accidentally acknowledge an incident, you can undo it by clicking the More... button in the incident details page and selecting Unacknowledge Incident.

Unacknowledging an incident returns it to a Triggered state and causes notifications to be sent again. The escalation process also resumes.

Resolve an Incident

There are multiple ways to resolve PagerDuty incidents depending on your use case:

Redact an Incident

If an incident contains sensitive information, the Account Owner can permanently delete the incident's details by selecting More... and clicking the Redact Incident button.

After confirming that you want to redact an incident's name and details, the page reflects who redacted the data and when it was redacted.

FAQ

---