Sumo Logic设置.NET Web API日志事件超时告警方案咨询

Got it, let's break down exactly how to set up this alert to catch when your API action's start and finish logs are spaced more than 10 seconds apart. This relies on pairing logs for the same request, building a targeted query, and configuring the alert rules.

Step 1: Ensure Logs Include a Unique Request Identifier #

First, you need a way to link the "Started" and "Finished" logs to the same API request. In .NET, you can use the built-in Activity.Current.Id (from System.Diagnostics) or generate a custom request ID, and make sure this ID is included in both log entries.

Your logs should look something like this (adjust based on your logging format):

Started | RequestId: 7a9f2d4c-1b3e-5c79-8d0a-2b4f6d80e2c4 | Controller: OrderController | Action: ProcessOrder
Finished | RequestId: 7a9f2d4c-1b3e-5c79-8d0a-2b4f6d80e2c4 | Controller: OrderController | Action: ProcessOrder

If you're using structured logging (like Serilog or NLog), include the request ID as a structured field—this makes parsing in Sumo Logic way easier.

Step 2: Build the Sumo Logic Query #

Next, create a query that groups logs by request ID, calculates the time between "Started" and "Finished", and filters for durations over 10 seconds.

Example Query (Adjust for Your Log Format) #

// Replace with your actual source category or filter for your API logs
_sourceCategory=prod/dotnet/api-logs
// Parse the request ID and log mark (Started/Finished) from your logs
// If using JSON logs, use `json "RequestId" as requestId` instead
| parse "RequestId: * " as requestId
| parse "Mark: * " as logMark
// Group logs by request ID, keep groups where logs are within 1 hour (adjust as needed)
| transaction requestId maxspan=1h
// Only keep groups that have both a Started and Finished log
| where count=2
// Calculate duration in milliseconds (_endTime and _startTime are in epoch ms)
| calculate duration = _endTime - _startTime
// Filter for durations over 10 seconds (10,000 ms)
| where duration > 10000
// Show relevant fields for debugging
| fields requestId, duration, _startTime, _endTime, Controller, Action

Key Notes for the Query: #

• Parsing: Tweak the parse lines to match your actual log structure. If your logs are JSON, use Sumo's JSON parsing functions instead of regex-based parse.
• maxspan: Set this to a value larger than your longest expected normal API execution time (e.g., 1 hour) to avoid splitting logs for the same request into separate groups.
• Handling Incomplete Requests: If you want to alert on requests that have a "Started" log but no "Finished" (e.g., the API crashed), add an extra clause: | where count=1 OR duration > 10000.

Step 3: Configure the Alert in Sumo Logic #

Once your query works and returns the slow requests you want to catch:

1. Go to the Alerts section in Sumo Logic and click Create Alert.
2. Select Custom Query as the data source, paste your query, and set the time range (e.g., "Last 5 minutes" if you want to check frequently).
3. Set the Trigger Condition: Choose "When results are returned" (since any result means a request exceeded 10 seconds). You can also set a threshold like "When number of results is greater than 0".
4. Configure Notification Settings: Choose how you want to be alerted—email, Slack, Microsoft Teams, webhook, etc. Customize the message to include details like the request ID, duration, and action name.
5. Set Alert Frequency: Decide how often Sumo Logic runs the query (e.g., every 1 minute) and how often to send repeat notifications (e.g., every 5 minutes until the issue is resolved).
6. Name your alert clearly (e.g., "API Action Execution Exceeds 10 Seconds") and add a description for your team.

Step 4: Test the Alert #

To make sure everything works:

• Add a delay in your API action (e.g., Thread.Sleep(11000); in C#) and trigger a request.
• Wait for Sumo Logic to process the logs and run the alert query.
• Verify that the alert triggers and you receive the notification with the correct details.

Bonus Tips #

• Timezone Alignment: Ensure your logs use UTC timestamps to avoid timezone mismatches in Sumo Logic's time calculations.
• Structured Logging: Using structured logs (JSON) makes parsing and querying far more reliable than unstructured text logs.
• Adjust Thresholds: If 10 seconds is too strict or lenient, tweak the duration > 10000 value to match your API's performance requirements.

内容的提问来源于stack exchange，提问作者Sibtain Norain