The Safety Net

Before I deploy a single Conditional Access policy, before I configure a named location, before I touch anything in Entra ID, I build the safety net.

That might seem backwards. Most people think about alerting as something you layer on after the fact, an afterthought, a nice-to-have. That mentality is exactly why most environments have exclusions that outlive their purpose by months, breakglass accounts nobody's watching, and CA gaps that quietly sit open until something goes wrong.

It is almost impossible to properly monitor and secure Microsoft 365 without Azure. Not everyone has E5. Not every client has P2 licensing across the board. But Azure has offered a legitimately low-cost path to advanced automation for years, and most people aren't using it.

A well-configured Log Analytics Workspace costs pennies at SMB scale. Azure Automation Accounts are nearly free at low runbook execution volumes. The gap between "I wish I knew about this sooner" and "I know about this in real time" is often a $15/month Log Analytics workspace and a few Bicep templates.

In my case the safety net wasn't a single tool or a single alert. It was an Infrastructure as Code system that went in before any workload went live. The foundation was a Log Analytics Workspace paired with Blob Storage for log retention. From there:

Azure Automation Accounts act as the execution engine for scheduled and event-driven runbooks. Data Collection Rules route the right signals from the right sources into LAW. KQL queries and scheduled monitors turn raw log data into actionable alerts. Azure Runbooks (PowerShell) automate the logic that humans reliably forget to run manually. Key Vault stores the credentials and secrets everything else needs to operate.

When I say I deploy this before touching CA, I mean it literally. The Bicep alerting package puts over 68 alert rules in place before anything else. Infrastructure health, resource hygiene, and the identity-critical events that matter for CA.

Orphaned resources aren't directly CA-related, but they matter. They're noise that hides real signals. The identity-critical column is the part that ties directly to CA.

Every layer in the CA stack has exclusions. Every exclusion is a gap. Some gaps are intentional: breakglass accounts are excluded from MFA by design. Some are operational: a user gets added to CA-GlobalExclusions because they're having an auth issue and someone will "fix it later." Some are forgotten, like that exclusion from six months ago that nobody removed.

One example that ties directly to the previous post: Device Code Flow. Blocking it with a CA policy is the right move, but blocking it doesn't mean you'll know when someone tried to use it before the block went in, or when an exclusion opens the door back up. The signal exists in sign-in logs via AuthenticationProtocol == deviceCode.

The most dangerous thing in a CA environment isn't a missing policy. It's an exclusion that stopped being temporary.

I had an Azure Runbook on a schedule, daily at minimum and multiple times a day for sensitive groups, checking whether any user had been sitting in CA-GlobalExclusions longer than the threshold. When it found one, it didn't just log it. It sent an alert.

The operational reality is that location-based CA policies will block users who travel, and the solution most people reach for is adding them to a traveling users group that bypasses the Named Locations block. That's fine. That's what it's for. But how do you know when someone is in that group? How do you make sure they come out of it when they get home?

Same pattern: a Runbook, a schedule, an alert.

We will create gaps. Users call with auth issues at 7pm. Someone adds them to an exclusion group to unblock them. They get busy. The exclusion stays. Three months later that account gets compromised and you're trying to figure out why MFA didn't fire.

The CA policies are the architecture. The safety net is what catches the architecture failing at 7pm on a Friday.

The automation fills the gap between what you intend and what you actually remember to check at 9pm on a Tuesday. Nobody catches everything manually. That's not a criticism, it's just true.

You don't need E5 to do this.

Log Analytics, Automation Accounts, Data Collection Rules, Key Vault: these are Azure resources. At SMB scale, most of the components stay within free tier limits. The ones that don't run well under $50/month total.

The gap between "I can't afford E5" and "I have zero visibility" is mostly a willingness-to-build problem, not a budget problem.

We started at the top of the CA stack and worked down: MFA, admins, locations, Device Code Flow, Legacy Auth. The safety net was always the missing piece at the bottom.

Every policy you deploy creates potential gaps. Every exclusion is a calculated risk. The safety net doesn't eliminate them. It tells you when they're being used.

Build the net first. Then build the stack on top of it.

Jon Hope

Microsoft MVP · Security: Identity & Access