Back to writing
4 min readDetection Engineering

Evaluating Identity Detection Coverage Across MSP Tenants

A bounded Microsoft Entra evaluation method, with synthetic cases for correlation and tenant separation.

By Kevin O'Connor

A shared identity dashboard can look reassuring while one customer has no usable sign-in history and another customer's alerts route to the wrong queue. For an MSP, coverage is a property of each tenant's telemetry, permissions, detection rules, and response path. A consolidated screen is only a view of those dependencies.

This article focuses on Microsoft Entra workforce tenants with a possible Microsoft Sentinel analytics layer. It compares documented roles within that stack and supplies a local synthetic correlation exercise. It does not report a production ITDR deployment, a permissioned Microsoft tenant test, or a ranking of competing vendors. Documentation was reviewed on September 9, 2026; the local experiment uses no customer data or cloud account.

Separate signal production from case handling

Entra sign-in and audit records describe identity activity. Entra ID Protection adds Microsoft-generated risk information. Sentinel can collect selected telemetry and support additional analytics and incidents. Those layers can complement each other, but they aren't interchangeable.

LayerWhat to evaluateWhat an empty result cannot establish
Entra activity logsRequired interactive and non-interactive sign-ins, workload identities, directory changes, actor and target IDsThat every relevant identity action was collected
Entra ID ProtectionLicensed risk details, the applicable detection type, detection timing, and remediation stateThat a custom administrative misuse scenario would receive a native risk detection
Sentinel analyticsConnector health, normalization, scheduled-query behavior, incident ownership, and response permissionsThat the original identity provider blocked an action

Microsoft documents P2 requirements for full risky-user data. It also distinguishes the roles needed to read reports from those needed to act on users. Build those prerequisites into onboarding. A successful administrator login to a portal is a poor substitute for checking the actual data and operations your analysts need.

Start with a tenant manifest

For each invented lab tenant, and later each authorized customer, record its immutable tenant ID, workspace resource ID, responsible customer contact, delegated analyst roles, connector configuration, expected sources, retention, and response authority. Keep this manifest under change control. Domain names and display names are useful labels but weak correlation keys.

Microsoft's Entra retention table currently lists seven days for standard audit and sign-in logs on Free and 30 days on P1/P2. Risk reports have separate retention. Expired historical records do not return when a license is upgraded. Longer analysis windows therefore require a planned destination and retention policy before the event occurs.

Check freshness with the last source timestamp and ingestion timestamp for every expected category. A rule that returns no matches should have a separate data-health result. Otherwise a broken connector can make a tenant look unusually quiet.

Test tenant boundaries in the whole workflow

Microsoft describes customer workspaces managed through Azure Lighthouse as a way to retain data in the customer tenant while delegating management. That architecture helps define custody, but exported CSV files, central tickets, and automation outputs can still move sensitive data outside the workspace.

A hypothetical tenant A and tenant B should both contain a synthetic identity with the same local identifier. Give A failed sign-ins and B a later role assignment. The combined view must not invent a cross-customer attack chain. Tenant binding belongs in joins, suppression keys, deduplication keys, incident metadata, and remediation commands.

Microsoft also documents that alerts from cross-workspace analytics rules exist in the workspace where the rule is defined. Decide who owns that central incident, how the affected customer is identified, and whether the response identity is authorized in that customer's tenant. Test a revoked delegation: the system should surface the failed action and its target, rather than closing a case as remediated.

What the local cases measured

Download identity_hunt.py and run it with Python 3. It creates an in-memory SQLite database from 35 synthetic records and exercises eight cases. The hypothesis is a burst of failed sign-ins, a success, and a subsequent role grant by the same actor. Details and the runnable SQL are in the companion hunt article.

The recorded run produced two intended matches. An ordinary administrator sequence, a two-failure sequence, a grant outside the window, cross-tenant correlation, duplicate delivery, and missing success telemetry produced no matches. The result file gives expected and observed outcomes.

The missing-telemetry case is a deliberate warning about interpretation: a negative query result can be a missed sequence. These checks validate the local query over an invented schema. They don't measure Entra detection sensitivity, Sentinel ingestion delay, a provider's alert quality, or analyst permissions. Those remain acceptance tests for a separately authorized Microsoft lab.

Price the coverage you intend to operate

Request a quote against a workload, not a tenant count alone. Include licensed users and workload identities, daily ingested bytes by source, required retention, rule execution, automation, and expected analyst handling time. Measure source volume in the authorized pilot, then estimate quiet-day and burst-day costs using the applicable regional billing terms. I haven't measured those volumes or obtained vendor quotes here.

Use the same cases to measure detection delay from source time, enrichment quality, duplicated incidents, analyst handling time, and successful response acknowledgement. A cheaper option with a large onboarding or handling burden may be the more expensive operating model.

I would advance a candidate after every tenant has visible telemetry health, the negative isolation cases pass, and a response exercise reaches the intended customer with an auditable outcome. Until that happens, the comparison describes a plausible architecture with a local query check, not demonstrated tenant coverage.

Email updates

Get new research by email

In-depth notes on AI security, threat research, and practical defensive work.

To unsubscribe, email kevin@kevinbytes.com.