Pending versus firing alerts

Avoid paging on a momentary spike

Lesson 12 of 20 · 12 min · Module 4: A plan and actionable alerts

A sustained condition and a brief spike can require different responses.

Start this lesson →

Free. No installation: you write and run the code in your browser, and it is checked automatically.

By the end, you’ll be able to…

  • Name the timing requirement
  • Distinguish pending from healthy

The idea

An alert combines a defined metric, a condition, and a timing requirement. Its state should follow the supplied evidence, rather than the presence of a rule name.

01Name the timing requirement

The threshold is exceeded, but the condition has not yet lasted for the configured hold duration. Both value and duration matter.

02Distinguish pending from healthy

Pending does not mean the metric is below threshold. It means the firing condition has not yet satisfied its persistence requirement.

03Choose response times consciously

A long hold reduces noise but can delay recovery work. Choose it from impact, measurement behavior, and operational response goals.

Your turn

Use HighErrorRatio with expr http_error_ratio > 0.05 and for 5m. This scenario has lasted only 60 seconds, so the rule must be pending rather than firing.

What the checks look for

  • Keeps the intended hold and expression
  • Records pending rather than firing
Try it in the editor →

Take this with you

Use timing as part of an explicit response policy.

Go a little further

Compare a short transient metric sample with a sustained service outage.

Learn Infrastructure & Operations for free

20 short lessons with real code and instant checks.

See the whole course →