Leveraging Summary Rules in Microsoft Sentinel: A Practical Guide

Introduction

In today's security landscape, organisations face an overwhelming volume of security data. Microsoft Sentinel, as a cloud-native SIEM solution, processes vast amounts of logs daily. However, the sheer volume can impact both performance and cost-efficiency. This is where Summary Rules come into play—a powerful feature that helps security teams distil raw data into actionable insights while optimising resources.

Thanks to Bert-Jan Pals and Fabian Bader for their awesome talk on this at Experts Live DK earlier in the year and Bert-Jan Pals for some of the summary rules with me.

What Are Summary Rules?

Summary Rules in Microsoft Sentinel are scheduled KQL queries that transform high-volume raw data into concise, aggregated datasets. These rules process data at regular intervals, creating summarised tables that are both more manageable and more cost-effective to query during investigations.

Why Use Summary Rules?

There are several compelling reasons to implement Summary Rules in your Sentinel environment:

1. Cost Reduction: By aggregating data, you reduce the volume of logs that need to be queried during investigations, potentially lowering Log Analytics costs.
2. Performance Improvement: Queries run faster against summarised data, accelerating your security team's response times.
3. Extended Data Retention: Summary tables can retain important information beyond your standard retention period, giving you access to historical patterns without the cost of storing all raw data.
4. Streamlined Analysis: Pre-aggregated data makes it easier to identify trends, anomalies, and patterns that might be obscured in raw logs.

Cost Optimisation with Basic/Auxiliary Logs and Summary Rules

One particularly powerful cost-saving strategy involves the strategic use of Summary Rules in conjunction with Microsoft Sentinel's tiered data storage options. This approach allows organisations to dramatically reduce their analytics costs whilst maintaining critical security visibility.

The Tiered Data Approach

Microsoft Sentinel offers different tiers for log storage—Analytics, Basic, and Auxiliary. Analytics logs provide full query capabilities but at the highest cost. Basic logs offer limited query capabilities at a significantly lower cost. Auxiliary logs are even more cost-effective but have the most query limitations.

For high-volume data sources that generate substantial logs but where only specific patterns or anomalies are of interest, you can implement a sophisticated data management strategy:

First, route your high-volume data to Basic or Auxiliary log tiers instead of the more expensive Analytics tier. This immediate shift can result in substantial cost savings—potentially reducing storage costs by 50-80% depending on your data profile.

Next, create Summary Rules that periodically query these Basic or Auxiliary logs, extracting only the essential security-relevant information and patterns. These Summary Rules transform the raw data into highly concentrated security intelligence.

Finally, store these summarised results in dedicated Analytics tables that are specifically designed for security detections. Because these tables contain only the essential security information—not the entire raw dataset—their size is dramatically smaller, making them much more cost-effective to maintain in the Analytics tier.

Real-World Application

For example, consider authentication logs that might generate millions of events daily. Rather than storing all these events in the Analytics tier:

1. Route the raw authentication logs to the Basic tier
2. Create a Summary Rule that identifies only unusual authentication patterns, such as first-time logins from new locations or after-hours access attempts
3. Store these concentrated findings in a dedicated Analytics table named "AuthenticationAnomalies"
4. Run your detection rules against this much smaller, focused dataset

This approach ensures you maintain visibility into security-relevant events while potentially reducing your analytics costs by orders of magnitude.

How Summary Rules Work

Summary Rules operate on a defined schedule, running KQL queries against your raw data and storing the results in custom tables. These tables become valuable resources for both automated detections and manual investigations.

The process begins with crafting a KQL query that effectively summarises the specific data patterns you want to track. This query forms the foundation of your Summary Rule and determines what insights you'll extract from your raw data. Once the query is defined, you configure the rule to run at specified intervals—whether hourly, daily, or weekly—based on your security monitoring requirements and the volatility of the data being tracked.

When the rule executes on schedule, Microsoft Sentinel processes the query and stores the results in a custom table that you've named. This table structure preserves your aggregated data in an optimised format, ready for rapid analysis. From this point forward, your security operations can leverage this summarised data through analytics rules that trigger alerts on specific patterns, or through interactive workbooks that visualise trends and anomalies. The entire process transforms overwhelming volumes of raw logs into focused, actionable security intelligence.

Practical Examples of Summary Rules

Let's explore some practical examples of how Summary Rules can be implemented in Microsoft Sentinel:

Example 1: Microsoft Graph API Activity Monitoring

MicrosoftGraphActivityLogs 
| where ingestion_time() > ago(35m) 
| extend ObjectId = coalesce(ServicePrincipalId, UserId) 
| extend ObjectType = iff(isempty(UserId), "ServicePrincipalId", "UserId") 
| where RequestUri !has "microsoft.graph.delta" 
| extend NormalizedRequestUri = tostring(parse_url(RequestUri).Path) 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}', @'<UUID>') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'[a-zA-Z0-9_-]{41,65}', @'<ID>') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'\d+$', @'<ID>') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'\/+', @'/') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'https:\/', @'https://') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'%23EXT%23', @'') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'\/[a-zA-Z0-9+_.\-]+@[a-zA-Z0-9.]+\/', @'/<UPN>/') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'^\/<UUID>', @'') 
| extend NormalizedRequestUri = replace_regex(NormalizedRequestUri, @'\?.*$', @'') 
| summarize GraphEndpointsCalled = make_set(NormalizedRequestUri, 1000), IPAddresses = make_set(IPAddress) by ObjectId, ObjectType

Explanation: This summary rule aggregates Microsoft Graph API activities, normalising the request URIs to remove sensitive information and unique identifiers. The resulting dataset provides security teams with a clean view of which endpoints are being accessed by which identities and from which IP addresses. This is invaluable for detecting unusual API access patterns that might indicate compromised credentials or service principals.

Example 2: Group Membership Tracking

let StartDate = startofday(ago(1d));
let EndDate = startofday(now());
IdentityInfo
| summarize arg_max(TimeGenerated, *) by AccountObjectId
| mv-expand GroupMembership
| summarize TotalMemberships = dcount(tostring(GroupMembership)), MemberOf = make_set(tostring(GroupMembership), 1000) by AccountObjectId, AccountDisplayName, AccountUPN
| extend ReportDate = now()

Explanation: This rule creates a daily snapshot of all user group memberships across the organisation. By storing this information in a summary table, security teams can easily track group membership changes over time and detect unusual additions to privileged groups. This capability is crucial for identifying potential privilege escalation attacks or insider threats.

Example 3: Role Assignment Monitoring

let StartDate = startofday(ago(1d));
let EndDate = startofday(now());
IdentityInfo
| summarize arg_max(TimeGenerated, *) by AccountObjectId
| mv-expand AssignedRoles
| where isnotempty(AssignedRoles)
| summarize TotalRoles = dcount(tostring(AssignedRoles)), Roles = make_set(tostring(AssignedRoles), 100) by AccountObjectId, AccountDisplayName, AccountUPN
| extend ReportDate = now()

Explanation: Similar to the group membership example, this rule tracks role assignments across identities. It provides a comprehensive view of who has which roles, making it easier to monitor privileged access and ensure proper segregation of duties. By comparing data over time, security teams can quickly identify suspicious changes to role assignments that might indicate compromise.

Best Practices for Implementing Summary Rules

When implementing Summary Rules in your environment, consider these best practices:

1. Focus on High-Volume Tables: Apply summary rules to tables with large volumes of data where patterns and aggregations are valuable.
2. Choose Appropriate Intervals: Set the rule frequency based on your security requirements—more frequent for critical data, less frequent for historical trending.
3. Use Efficient KQL: Optimise your queries to ensure they run efficiently within the scheduled timeframe.
4. Monitor Performance: Regularly review the execution times and resource consumption of your summary rules.
5. Align with Security Use Cases: Design summary rules that directly support your detection and investigation workflows.

Conclusion

Summary Rules represent a powerful capability within Microsoft Sentinel that enables security teams to work smarter, not harder. By transforming raw data into meaningful aggregations, they improve both the efficiency and effectiveness of security operations.

By implementing the example rules discussed in this blog, you can begin to realise the benefits of Summary Rules in your own environment—reducing costs, improving performance, and gaining new insights into your security posture.

Remember that the most effective Summary Rules are those tailored to your organisation's specific security needs and data volumes. Start with these examples, then adapt and expand as you discover the patterns most relevant to your security monitoring requirements.

SPONSORED

I hope you've found this guide helpful in enhancing your security posture. If you've enjoyed this content and would like to support more like it, please consider joining the Supporters Tier. Your support helps me continue creating practical security automation content for the community.

Supporters Tier Upgrade