使用log4net.Appender.AzureAppendBlobAppender遇BlockCountExceedsLimit异常求助

Hey Karthik, sorry to hear you're stuck with this block limit problem—let's walk through practical fixes and alternatives to get your logging back on track without hitting that 50000 block cap.

Fixes for the Existing AzureAppendBlobAppender #

1. Batch Multiple Log Entries into a Single Block #

The root issue here is that each log event is being sent as a separate block. If you bundle multiple logs into one block before submitting, you'll drastically reduce the number of blocks used. For example, bundling 512 logs into one block turns 50000 blocks into support for 25.6 million logs—way more than most daily needs.

You can create a custom appender that inherits from AzureAppendBlobAppender and adds batching logic:

public class BatchedAzureAppendBlobAppender : AzureAppendBlobAppender
{
    private readonly List<string> _pendingLogs = new List<string>();
    // Configure this via log4net settings if needed
    private int _batchSize = 512;

    protected override void Append(LoggingEvent loggingEvent)
    {
        lock (_pendingLogs)
        {
            // Render the log event to string like the base appender does
            var logLine = RenderLoggingEvent(loggingEvent);
            _pendingLogs.Add(logLine);

            // Submit when we hit the batch size
            if (_pendingLogs.Count >= _batchSize)
            {
                var combinedLogs = string.Join(Environment.NewLine, _pendingLogs) + Environment.NewLine;
                // Reuse the base appender's blob append logic
                AppendToBlob(combinedLogs);
                _pendingLogs.Clear();
            }
        }
    }

    // Make sure we don't lose pending logs when the app shuts down
    protected override void OnClose()
    {
        lock (_pendingLogs)
        {
            if (_pendingLogs.Count > 0)
            {
                var combinedLogs = string.Join(Environment.NewLine, _pendingLogs) + Environment.NewLine;
                AppendToBlob(combinedLogs);
                _pendingLogs.Clear();
            }
        }
        base.OnClose();
    }
}

Then update your log4net config to use this custom appender instead of the original AzureAppendBlobAppender.

2. Automatically Rotate Append Blobs #

Another approach is to switch to a new append blob once the current one hits the block limit. For example, name blobs with a date suffix (like app-logs-2024-05-20.txt) and create a new blob each day, or when the block count approaches 50000.

To implement this, extend the appender to check the current blob's block count periodically. Note: Fetching the block list adds a small overhead, so combining this with batching (from the first fix) is ideal to minimize checks.

Alternative Logging Approaches #

If you want to avoid modifying the appender entirely, consider these cloud-friendly alternatives:

Use Azure Application Insights #

Application Insights is built for cloud app monitoring and has no block limits. It integrates seamlessly with log4net via the log4net.Appender.ApplicationInsightsAppender. You get built-in log querying, error alerts, and performance tracking—way more than just blob storage.

Switch to Serilog with Azure Blob Sink #

Serilog is a modern logging library with better cloud integrations out of the box. The Serilog.Sinks.AzureBlobStorage sink automatically batches log entries and supports blob rotation, so you won't hit the block count limit. It's easy to set up and plays nicely with .NET web apps.

Use Azure Log Analytics Workspace #

Send logs directly to Log Analytics, which is designed for large-scale log ingestion and analysis. There's a log4net appender for Log Analytics, and it handles scaling and storage limits automatically. You can correlate logs with other Azure service data for deeper insights.

Switch to Block Blobs Instead #

If you want to stick with blob storage, use block blobs instead of append blobs. You can write batches of logs to a block blob, and when it reaches a certain size, commit the blocks and create a new blob. This requires more manual management, but avoids the append blob block limit.

内容的提问来源于stack exchange，提问作者Karthik