A query can parse correctly, use real column names, and still miss the behavior it claims to detect. It may select the wrong activity, merge unrelated identities, miss delayed events, or create an alert that an analyst cannot investigate. Treat a hunting query as an input to detection engineering, with a separate promotion record and acceptance tests. This library’s documentation and static checks do not replace tenant execution.
Write the detection contract before tuning
State the observable hypothesis precisely. “A successful service-principal sign-in followed a credential change” is measurable; “the new credential was used maliciously” may not be supported by those records. Identify which facts are observed, which are inferred, and what additional evidence would strengthen the conclusion.
Our contract includes data sources and collection mode, required table plan, directory and object keys, event-time window, ingestion allowance, aggregation unit, threshold rationale, expected benign scenarios, and an analyst action. Specify whether a count represents raw rows, unique source events, expanded targets, or distinct principals. Record how duplicates and absent fields affect the result.
Read the current table schema, then inspect representative tenant payloads. A dynamic column can exist while a nested property is absent or shaped differently for a particular operation. An activity name observed in one connector is not automatically portable to another source with similar terminology.
Separate four validation layers
| Layer | What it establishes | What it does not establish |
|---|---|---|
| Static parse and schema analysis | Syntax, known columns, compatible types | Actual payload meaning or useful detection |
| Synthetic semantic cases | Expected behavior for controlled inputs | Real connector coverage or production noise |
| Tenant replay and benign exercise | Runtime behavior on available data | Guaranteed coverage of future attacks |
| Operational pilot | Scheduling, alert handling, and analyst usefulness | A permanent guarantee after schema or policy changes |
Build fixtures for a positive case, a near miss, an allowed maintenance action, a missing identifier, duplicate delivery, a cross-tenant name collision, and a delayed event. Prefer stable object keys over display names. For joins, assert expected cardinality: one credential event with several later sign-ins must not disappear through an unintended deduplication or expand into unexplained duplicates.
Parameterize the evaluation time for fixture testing. Repeated use of live now() makes boundary failures difficult to reproduce. Keep baseline and investigation intervals nonoverlapping when the test is meant to identify unfamiliar activity. Include a test where the baseline is unavailable rather than interpreting missing history as proven novelty.
Engineer the schedule around observed delay
Scheduled analytics rules have independent query frequency and lookback settings. Overlapping windows can repeat results. Microsoft also documents a scheduling delay and a results simulation over previous executions; simulation evaluates available data, not an independent labeled attack corpus. Scheduled-rule semantics.
Measure ingestion delay per source and include the actual tail, not just an average. Microsoft’s guidance combines a wider event-time window with an ingestion-time restriction to address a single-stream delay problem. For multi-source correlation, measure each side and test records arriving in opposite orders. Ingestion-delay guidance.
Our recommended correlation test includes a pair whose first record was ingested in the previous run and whose second record arrives now. Filtering both inputs to only newly ingested rows can lose that pair. Define an anchor or reconciliation design and a stable detection key, then document any residual late-arrival limit. Do not present ingestion_time() as a precise source clock or a way to observe events that never arrived.
Convert a hunt into an alert deliberately
Remove exploratory output limits only after estimating result volume and validating the intended schedule. A take 200 cap limits returned rows, not the preceding data scan, and can hide events when copied blindly into a rule. Ensure the final query preserves the fields required by the selected rule engine and maps entities using correct identifiers.
Choose alert grouping and incident behavior for the investigation, not simply the default UI setting. Scheduled rules have documented per-run alert limits, and grouping choices change how multiple results are represented. Test a high-volume case so the analyst understands aggregation and truncation behavior. Alert grouping and limits.
Run a pilot with an owner, a review interval, and recorded dispositions. Measure useful findings, repeated benign causes, missing evidence, and time to investigate. Tune a specific actor/resource relationship with an expiring exception rather than suppressing every event from a powerful deployment identity.
Monitor the detection as a production system
Rule execution can fail because of permissions, missing functions, resource limits, or mapping problems. Sentinel provides health and audit evidence, including recommended helper functions, to monitor execution and changes. An empty result is not evidence that a failed query found nothing suspicious. Rule health and integrity.
Use supported historical rerun and execution-insight facilities to investigate changes, while remembering that retained data and current query behavior can differ from the original run. Keep query version, rule settings, test corpus version, and evaluation interval together. Execution validation tools.
Before promotion, require a passing semantic matrix, a benign tenant exercise, confirmed collection coverage, acceptable runtime, correct entity mapping, and an analyst-readable investigation path. Repeat validation when connectors, table plans, identity mappings, or detection logic change. Continue with correlation engineering and the query library for source-specific starting points.
Go to the source