Introduction

Every time a problem happens in production, the first thing we do is open the logs. But how many times have you found something like this?

1
Erro ao processar pedido

No OrderId. No CustomerId. No way of knowing which instance it happened on or which version of the application was running. This kind of log exists in thousands of applications β€” and it helps absolutely no one.

The problem is not a lack of logs. It is a lack of context, a lack of structure and a lack of control. When the application needs more detail, you have to change the configuration, deploy, wait for the restart β€” and hope you capture the error before rolling back.

In this article, I will show a complete logging strategy for .NET 8 that solves these three problems at once:

  • Structured: each log entry carries typed properties, not just plain text
  • Contextual: every log line automatically includes information about the application, the host and the request
  • Dynamic: the log level can be changed in production with an automatic rollback timer β€” no deploy, no restart

All of this using native ILogger<T>, without unnecessary wrappers, compliant with SonarQube and integrated with Azure Application Insights.

The complete implementation is available in the blog-zocateli-sample repository on GitHub. Throughout the article, I will show the key snippets and explain the reasoning behind each decision. If you want to see the full context at any point, check the sources in the repository.

πŸ“¦ Source code: The complete implementation of this article is in the blog-zocateli-sample repository on GitHub. Clone it, explore it and adapt it to your context.

πŸ’‘ This article complements the .NET 8+ Configuration Pipeline, which explains how the configuration system, IOptions<T> and secrets work. The logging strategy described here depends directly on those concepts.


The Problem: Why Your Logs Don’t Help

Consider these two real production log scenarios:

Before β€” log without context:

1
2
3
info: Pedido processado
warn: Falha ao validar item
error: Erro ao processar pedido

A support analyst looks at this and cannot answer a single question: which order? Which customer? On which instance? Was it the new version or the old one? Does this error reproduce or was it a one-off?

After β€” structured log with context:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
{
  "Timestamp": "2026-03-07T14:32:18.123Z",
  "Level": "Error",
  "Message": "Failed to process order 3fa85f64-5717-4562-b3fc-2c963f66afa6 for customer C-1234",
  "OrderId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "CustomerId": "C-1234",
  "ApplicationName": "SampleApi",
  "ApplicationVersion": "1.0.0",
  "HostName": "pod-api-7b4f9c",
  "Environment": "Production",
  "CorrelationId": "0HN6KQ2JH2S5E:00000001",
  "EventId": 2004
}

With this second format, the analyst can:

  • Filter by OrderId to see everything that happened to that order
  • Group by HostName to identify whether the problem is specific to one instance
  • Use the CorrelationId to trace the whole pipeline of a single request
  • Compare ApplicationVersion to know whether the bug comes from the new version

The cost of troubleshooting drops drastically. Instead of hours digging through generic logs, you find the cause in minutes with a query in Application Insights.

πŸ’‘ A log without context in production is like searching Google for an error without typing the error message β€” technically you are searching, but you will not find anything useful.


Structured Logging: What It Is and Why It Matters

Structured logging means that each log entry carries typed, indexable properties, not just a concatenated string.

In .NET, the difference is subtle in the code but huge in the result:

1
2
3
4
5
// βœ… Structured β€” "OrderId" becomes a separate property
_logger.LogInformation("Order {OrderId} created for customer {CustomerId}", orderId, customerId);

// ❌ Interpolation β€” loses the properties, becomes plain text
_logger.LogInformation($"Order {orderId} created for customer {customerId}");

When you use the template with {OrderId}, ILogger preserves the value as a separate property. In Application Insights, it appears in customDimensions and can be filtered, aggregated and correlated with KQL queries. When you use string interpolation with $, the value is embedded in the message β€” Application Insights receives only text and cannot index it.

The practical consequences are straightforward:

  • With structured properties: customDimensions["OrderId"] == "3fa85f64..." β€” filterable
  • With an interpolated string: you need contains or a regex on the message β€” slow and imprecise

Beyond the functional issue, SonarQube flags string interpolation in logs as rule S6664 (Log message template should be compile-time constant), since it can hurt performance and you lose the ability to group similar messages.

⚠️ Never use $"Order {orderId}" in log calls. It creates a new string on every call (even when the log level is disabled) and loses the structured property. Always use the template: "Order {OrderId}".


Automatic Enrichment: Application Context in Every Log

If every developer has to remember to include ApplicationName, HostName and CorrelationId in each log call, it is not going to happen. The solution is to enrich automatically.

Middleware with ILogger.BeginScope()

ILogger supports scopes β€” a block that adds properties to every log entry made inside it. By combining this with a middleware, we guarantee that every log of any request carries the application context:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
public sealed class LogEnrichmentMiddleware : IMiddleware
{
    private readonly IOptions<LoggingOptions> _options;
    private readonly IHostEnvironment _environment;

    public LogEnrichmentMiddleware(IOptions<LoggingOptions> options, IHostEnvironment environment)
    {
        _options = options;
        _environment = environment;
    }

    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        var logger = context.RequestServices
            .GetRequiredService<ILogger<LogEnrichmentMiddleware>>();

        var enrichmentData = new Dictionary<string, object>
        {
            ["ApplicationName"] = _options.Value.ApplicationName,
            ["ApplicationVersion"] = _options.Value.Version,
            ["HostName"] = Environment.MachineName,
            ["Environment"] = _environment.EnvironmentName,
            ["CorrelationId"] = context.TraceIdentifier
        };

        using (logger.BeginScope(enrichmentData))
        {
            await next(context);
        }
    }
}

The middleware is registered as a singleton (because it implements IMiddleware) and enabled in the pipeline:

1
2
3
builder.Services.AddSingleton<LogEnrichmentMiddleware>();
// ...
app.UseMiddleware<LogEnrichmentMiddleware>();

The ApplicationName and Version properties come from a dedicated section of appsettings.json:

1
2
3
4
5
6
7
8
{
  "Logging": {
    "Application": {
      "ApplicationName": "SampleApi",
      "Version": "1.0.0"
    }
  }
}

ℹ️ The LoggingOptions class is configured through the IOptions<T> pattern. If you want to understand in depth how this mechanism works, see the article about the .NET 8+ Configuration Pipeline.

TelemetryInitializer for Application Insights

BeginScope() enriches the ILogger logs, but the telemetry Application Insights captures automatically (HTTP requests, dependencies, exceptions) does not go through ILogger. To ensure that all telemetry carries the same properties, we use an ITelemetryInitializer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
public sealed class ApplicationTelemetryInitializer : ITelemetryInitializer
{
    private readonly string _applicationName;
    private readonly string _version;
    private readonly string _hostName;
    private readonly string _environment;

    public ApplicationTelemetryInitializer(
        IOptions<LoggingOptions> options, IHostEnvironment env)
    {
        _applicationName = options.Value.ApplicationName;
        _version = options.Value.Version;
        _hostName = Environment.MachineName;
        _environment = env.EnvironmentName;
    }

    public void Initialize(ITelemetry telemetry)
    {
        telemetry.Context.GlobalProperties.TryAdd("ApplicationName", _applicationName);
        telemetry.Context.GlobalProperties.TryAdd("ApplicationVersion", _version);
        telemetry.Context.GlobalProperties.TryAdd("HostName", _hostName);
        telemetry.Context.GlobalProperties.TryAdd("Environment", _environment);
        telemetry.Context.Component.Version = _version;
    }
}

Registered in DI as:

1
builder.Services.AddSingleton<ITelemetryInitializer, ApplicationTelemetryInitializer>();

With these two pieces, you have full coverage: LogEnrichmentMiddleware for ILogger logs and ApplicationTelemetryInitializer for request traces, dependency calls and exceptions captured automatically by the SDK.


Dynamic Log Level with an Automatic Timer

A real scenario: your production application is behaving strangely. You need to see Debug-level logs to understand what is going on. But the production appsettings.json is configured with Information.

The traditional path would be: change the configuration, deploy (or restart), capture the detailed logs, and then undo everything so as not to drown Application Insights in volume (and cost). This process can take from minutes to hours depending on your pipeline.

The solution proposed here is an administrative endpoint that changes the log level at runtime, with an automatic rollback timer:

1
2
3
4
5
POST /api/admin/log-level
{
  "level": "Debug",
  "durationMinutes": 15
}

After 15 minutes, the level automatically returns to the one configured in appsettings.json. No deploy. No restart. No risk of forgetting Debug turned on.

The Mechanism: A Custom ConfigurationProvider

The .NET logging system reads log levels from the configuration pipeline. The key that defines the default level is Logging:LogLevel:Default, and per category it is Logging:LogLevel:{CategoryName}.

The strategy is to create a custom ConfigurationProvider that can be updated at runtime and that, when changed, triggers OnReload() to notify the framework:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
public sealed class DynamicLogLevelConfigurationProvider : ConfigurationProvider
{
    public override void Load()
    {
        // Starts empty β€” values are set dynamically via Update()
    }

    public void Update(string key, string value)
    {
        if (value is null)
        {
            Data.Remove(key);
        }
        else
        {
            Data[key] = value;
        }

        OnReload();
    }

    public void Clear()
    {
        Data.Clear();
        OnReload();
    }
}

The critical point is the registration: this provider must be added last in the configuration pipeline, so that its value overrides the one from appsettings.json:

1
2
3
// In Program.cs β€” AFTER CreateBuilder (which already adds appsettings)
((IConfigurationBuilder)builder.Configuration)
    .Add(new DynamicLogLevelConfigurationSource());

The Service with a Timer

DynamicLogLevelService orchestrates the change and automatic rollback logic:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
public sealed class DynamicLogLevelService : IDisposable
{
    private readonly DynamicLogLevelConfigurationProvider _provider;
    private readonly ILogger<DynamicLogLevelService> _logger;
    private readonly object _lock = new();
    private Timer _revertTimer;
    private LogLevelOverride _currentOverride;

    public DynamicLogLevelService(
        IConfiguration configuration,
        ILogger<DynamicLogLevelService> logger)
    {
        _logger = logger;
        var root = configuration as IConfigurationRoot
            ?? throw new InvalidOperationException("...");
        _provider = root.Providers
            .OfType<DynamicLogLevelConfigurationProvider>()
            .FirstOrDefault()
            ?? throw new InvalidOperationException("...");
    }

    public void SetLogLevel(
        string category, LogLevel level, TimeSpan duration)
    {
        ArgumentOutOfRangeException
            .ThrowIfLessThanOrEqual(duration, TimeSpan.Zero, nameof(duration));

        var key = $"Logging:LogLevel:{category ?? "Default"}";

        lock (_lock)
        {
            _revertTimer?.Dispose();
            _provider.Update(key, level.ToString());

            _currentOverride = new LogLevelOverride(
                Category: category,
                Level: level,
                ExpiresAt: DateTimeOffset.UtcNow.Add(duration));

            _revertTimer = new Timer(
                callback: _ => RevertToDefault(),
                state: null,
                dueTime: duration,
                period: Timeout.InfiniteTimeSpan);
        }
    }

    public void RevertToDefault()
    {
        lock (_lock)
        {
            _revertTimer?.Dispose();
            _revertTimer = null;
            _provider.Clear();
            _currentOverride = null;
        }
    }
}

Important points about this implementation:

  • The Timer is created with Timeout.InfiniteTimeSpan as the period β€” it runs only once after the dueTime
  • If SetLogLevel is called again before the timer expires, the previous timer is disposed and a new one is created
  • The lock guarantees thread safety between the timer (which runs on a ThreadPool thread) and concurrent calls
  • IDisposable is implemented for proper cleanup of the Timer

πŸ’‘ Protect this endpoint with proper authorization β€” in production, only administrators should change the log level. See the article about Authentication and Authorization with JWT, OAuth2 and OpenID for protection strategies.


Native ILogger: No Reinventing the Wheel

A pattern I see with worrying frequency is the creation of an ICustomLogger or LogService that wraps ILogger<T>:

1
2
3
4
5
6
// ❌ Anti-pattern β€” DO NOT do this
public class CustomLogger
{
    private readonly ILogger<CustomLogger> _logger;
    public void LogInfo(string message) => _logger.LogInformation(message);
}

This wrapper breaks several mechanisms:

  • The log category always becomes CustomLogger instead of the real class that originated the log
  • Namespace filters (Logging:LogLevel:MyNamespace) stop working
  • Traceability in Application Insights suffers
  • Tests become more complex (you need to mock the wrapper instead of ILogger)

The correct approach in modern .NET is to use ILogger<T> directly and the [LoggerMessage] attribute to generate the log methods through a source generator:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
public static partial class LogMessages
{
    [LoggerMessage(
        EventId = 2000,
        Level = LogLevel.Information,
        Message = "Order {OrderId} created for customer {CustomerId}, total: {Total}")]
    public static partial void OrderCreated(
        ILogger logger, Guid orderId, string customerId, decimal total);

    [LoggerMessage(
        EventId = 2003,
        Level = LogLevel.Debug,
        Message = "Processing order {OrderId}: validating {ItemCount} items for customer {CustomerId}")]
    public static partial void OrderProcessingDebug(
        ILogger logger, Guid orderId, int itemCount, string customerId);

    [LoggerMessage(
        EventId = 2004,
        Level = LogLevel.Error,
        Message = "Failed to process order {OrderId} for customer {CustomerId}")]
    public static partial void OrderFailed(
        ILogger logger, Exception exception, Guid orderId, string customerId);
}

Benefits of the [LoggerMessage] Source Generator

  1. Performance: the source generator creates code that checks the log level before building the message β€” zero allocation when the log is disabled
  2. AOT-friendly: compatible with Native AOT, no reflection at runtime
  3. SonarQube compliance: satisfies rules S6664 (compile-time constant template), S2629 (logging guard) and S6667 (structured logging)
  4. EventId: each message has a fixed numeric ID, which makes alerts and dashboards easier
  5. Testable: since they are static methods with ILogger as a parameter, passing a mock in tests is trivial

Usage is straightforward in any endpoint or service:

1
LogMessages.OrderCreated(logger, newOrder.Id, newOrder.CustomerId, newOrder.Total);

⚠️ Creating an ICustomLogger that wraps ILogger<T> adds a layer with no value, breaks traceability and makes testing harder. Use ILogger<T> directly with [LoggerMessage] β€” it is the correct abstraction.


Integration with Azure Application Insights

Application Insights is Azure’s APM (Application Performance Management) that automatically captures HTTP requests, dependencies (databases, HTTP clients, queues) and exceptions. The structured logging we implemented complements this automatic telemetry with business context.

Setup

The configuration is minimal β€” one line in Program.cs and the connection string in the configuration:

1
2
builder.Services.AddApplicationInsightsTelemetry();
builder.Services.AddSingleton<ITelemetryInitializer, ApplicationTelemetryInitializer>();
1
2
3
4
5
{
  "ApplicationInsights": {
    "ConnectionString": "InstrumentationKey=..."
  }
}

⚠️ Never put the connection string directly in the source code or in a committed appsettings.json. Use environment variables, Azure Key Vault or User Secrets for local development. See how to configure it in the article .NET 8+ Configuration Pipeline.

How the Structured Log Shows Up in Application Insights

When you log with [LoggerMessage] and structured properties, each property appears in customDimensions in Application Insights. Combined with the properties injected by LogEnrichmentMiddleware and ApplicationTelemetryInitializer, the result is:

ColumnValue
messageOrder 3fa85f64… created for customer C-1234, total: 150.00
customDimensions.OrderId3fa85f64-5717-4562-b3fc-2c963f66afa6
customDimensions.CustomerIdC-1234
customDimensions.Total150.00
customDimensions.ApplicationNameSampleApi
customDimensions.ApplicationVersion1.0.0
customDimensions.HostNamepod-api-7b4f9c
customDimensions.CorrelationId0HN6KQ2JH2S5E:00000001

KQL Queries for Troubleshooting

With these indexed properties, queries in Application Insights become surgical:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
// All logs for a specific order
traces
| where customDimensions["OrderId"] == "3fa85f64-5717-4562-b3fc-2c963f66afa6"
| project timestamp, message, severityLevel,
          tostring(customDimensions["ApplicationVersion"]),
          tostring(customDimensions["HostName"])
| order by timestamp asc

// Errors grouped by host (detects a problematic instance)
traces
| where severityLevel >= 3
| summarize ErrorCount = count() by tostring(customDimensions["HostName"])
| order by ErrorCount desc

// Correlate logs with the HTTP request
traces
| where customDimensions["CorrelationId"] == "0HN6KQ2JH2S5E:00000001"
| union requests
| where operation_Id == "0HN6KQ2JH2S5E:00000001"
| order by timestamp asc

ℹ️ Application Insights already captures HTTP requests, dependencies and exceptions automatically β€” structured logging complements it with the business context that only your application knows.


The Sample Application

The sample application implements every concept in this article in a .NET 8 Minimal API. The project structure organizes the logging components in a dedicated folder:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
dotnet-structured-logging-sample/
β”œβ”€β”€ src/SampleApi/
β”‚   β”œβ”€β”€ Program.cs                  # Composition: DI, middleware, endpoints
β”‚   β”œβ”€β”€ appsettings.json            # Configuration with the Logging:Application section
β”‚   β”œβ”€β”€ Logging/
β”‚   β”‚   β”œβ”€β”€ DynamicLogLevelConfigurationSource.cs
β”‚   β”‚   β”œβ”€β”€ DynamicLogLevelConfigurationProvider.cs
β”‚   β”‚   β”œβ”€β”€ DynamicLogLevelService.cs
β”‚   β”‚   β”œβ”€β”€ LogEnrichmentMiddleware.cs
β”‚   β”‚   β”œβ”€β”€ ApplicationTelemetryInitializer.cs
β”‚   β”‚   β”œβ”€β”€ LoggingOptions.cs
β”‚   β”‚   └── LogMessages.cs          # [LoggerMessage] source generator
β”‚   β”œβ”€β”€ Endpoints/
β”‚   β”‚   β”œβ”€β”€ LogLevelEndpoints.cs    # GET/POST/DELETE /api/admin/log-level
β”‚   β”‚   └── OrderEndpoints.cs       # POST/GET /api/orders (business example)
β”‚   └── Models/
β”‚       β”œβ”€β”€ SetLogLevelRequest.cs
β”‚       └── Order.cs
β”œβ”€β”€ tests/SampleApi.Tests/          # 13 unit tests with xUnit and NSubstitute
β”œβ”€β”€ README.md
β”œβ”€β”€ global.json                     # Pinned to SDK 8.0.418
└── SampleApi.sln

Architecture diagram showing the request flow through the enrichment middleware, ILogger with the LoggerMessage source generator, and Application Insights with customDimensions

The diagram above shows how the components connect: each HTTP request goes through LogEnrichmentMiddleware, which opens a BeginScope() with the context properties. Any log call made within that request β€” whether in endpoints, services or internal libraries β€” automatically inherits those properties. In parallel, ApplicationTelemetryInitializer ensures that Application Insights’ automatic telemetry (requests, dependencies) also carries the same information.

How to Run Locally

1
2
cd src/SampleApi
dotnet run

Test the dynamic log endpoint:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# Enable Debug for 15 minutes
curl -X POST http://localhost:5000/api/admin/log-level \
  -H "Content-Type: application/json" \
  -d '{"level": "Debug", "durationMinutes": 15}'

# Check the current status
curl http://localhost:5000/api/admin/log-level

# Manually revert to the default
curl -X DELETE http://localhost:5000/api/admin/log-level

The complete repository with detailed instructions is at github.com/lzocateli/blog-zocateli-sample β€” Logging.


Tips and Good Practices

Here are the practices I consider essential for a robust logging strategy:

  1. Use log categories deliberately: .NET uses the class namespace as the category (ILogger<OrderService> produces the category SampleApi.Endpoints.OrderService). This lets you filter by namespace in appsettings.json β€” for example, "SampleApi.Endpoints": "Debug" enables debug only in the endpoints without affecting the rest of the application.

  2. Never log sensitive data: names, CPFs (Brazilian taxpayer IDs), tokens, passwords and financial data must not appear in logs. Besides the security risk, Brazil’s LGPD requires the protection of personal data. If you need to trace a user, use an opaque identifier such as CustomerId.

  3. Control Application Insights cost: at high volume, Application Insights charges for data ingestion. Use adaptive sampling to reduce volume without losing visibility. The SDK’s default configuration already includes sampling β€” avoid disabling it without a cost calculation.

  4. Consistent EventId per message: each [LoggerMessage] should have a unique, fixed EventId. This lets you create alerts in Azure Monitor based on the EventId (more reliable than a regex on the message) and makes operations dashboards easier.

  5. Development environment with a JSON console: configure appsettings.Development.json to use the JsonConsoleFormatter with IncludeScopes: true. This shows the structured properties right in the terminal during development, without needing Application Insights.

  6. Test your logs: with [LoggerMessage] and ILogger<T>, it is trivial to verify in unit tests that the correct logs were emitted. The sample application includes middleware tests that verify the properties injected into the scope.

  7. Log administrative operations as Warning: log level changes, restarts and manual operations should be logged as Warning β€” visible by default and easy to audit.


Conclusion

Efficient logging is not about quantity β€” it is about quality and context. In this article, we covered three pillars that, together, turn logs from noise into a diagnostic tool:

  • Structure: typed properties with the [LoggerMessage] source generator instead of concatenated strings
  • Automatic context: LogEnrichmentMiddleware with BeginScope() and ApplicationTelemetryInitializer ensure that every log entry carries information about the application, the host and the request
  • Dynamic control: a custom ConfigurationProvider with OnReload() and a timer lets you enable Debug in production safely and temporarily

All of this works with .NET’s native ILogger<T> β€” no wrappers, no invented abstractions, no SonarQube violations. Application Insights receives the properties in customDimensions, enabling surgical KQL queries for troubleshooting.

If your application still uses _logger.LogInformation($"Processing order {id}"), now is a good time to evolve. Clone the sample repository, adapt it to your context and see the difference the next time you need to investigate a problem in production.

To work with logs in long-running background services, see how to apply this same strategy in .NET Worker and Background Service.


Read Also


References