Introduction
In enterprise applications with high processing volume, not everything can β or should β happen inside an HTTP request. Massive imports, consuming messaging queues, periodic synchronizations, sending notifications in batches, processing domain events β these scenarios require background processes that run continuously, independently of requests, and that survive restarts in a controlled way.
.NET 8+ offers a robust infrastructure for this: Worker Services and the BackgroundService class. Built on the Generic Host (IHost), they share the same dependency injection, configuration and logging pipeline as Web APIs β but without the overhead of Kestrel and the HTTP middleware. This means everything you already know about IOptions<T>, ILogger<T>, IServiceProvider and DI lifetimes applies directly to Workers.
In this article, I will explore the types of Workers, how the Singleton, Scoped and Transient lifetimes behave in this context (with pitfalls that catch many developers by surprise), and how Workers integrate natively with pagination of large volumes, messaging for decoupling, parallelism with Parallel and Tasks and asynchronous programming with async/await.
π‘ Tip: If your application currently uses a
Task.Run()inside a controller to “process in the background”, this article will show the correct, production-ready approach for that scenario.
π¦ Source code: The full implementation of this article is in the blog-zocateli-sample repository on GitHub. Clone it, explore it and adapt it to your context.
What Is a .NET Worker Service
A Worker Service is a .NET application without a web interface, designed to run background tasks continuously or periodically. It uses the Generic Host β the same infrastructure a Web API uses internally β but without the HTTP server (Kestrel).
Creating a Worker Service
| |
| |
| |
Generic Host vs WebApplication
| Aspect | Host.CreateApplicationBuilder | WebApplication.CreateBuilder |
|---|---|---|
| Configuration pipeline | β Identical | β Identical |
| Dependency injection | β Identical | β Identical |
| Logging | β Identical | β Identical |
| IOptions | β Identical | β Identical |
| HTTP server (Kestrel) | β Not included | β Included |
| Middleware pipeline | β Not applicable | β Included |
| Endpoints / Controllers | β Not applicable | β Included |
| Typical use | Background processing, queues | HTTP APIs, web apps |
βΉοΈ Info: Since the configuration pipeline is identical, everything explained in .NET 8+ Configuration Pipeline β provider order,
IOptions<T>, Docker secrets,AddKeyPerFileβ applies directly to Workers.
IHostedService vs BackgroundService
.NET offers two abstractions for background services. Understanding the difference is essential:
IHostedService β Full Control
| |
BackgroundService β Simplified Abstraction
| |
| Aspect | IHostedService | BackgroundService |
|---|---|---|
| Control | Manual StartAsync + StopAsync | ExecuteAsync with automatic loop |
| Timer/Polling | Implement manually with Timer | Use Task.Delay inside the loop |
| Ideal scenario | Complex initialization/cleanup | Continuous processing loops |
| CancellationToken | Received in StartAsync/StopAsync | Received in ExecuteAsync |
| Inherits from | Interface (IHostedService) | Abstract class (BackgroundService) |
β οΈ Warning: The
BackgroundService’sExecuteAsyncruns in the background. If it threw an unhandled exception before .NET 8, the host kept running silently with the worker dead. Starting with .NET 8, unhandled exceptions inExecuteAsyncstop the entire host by default (behavior configurable viaHostOptions.BackgroundServiceExceptionBehavior).
Types of Workers and Their Use Cases
In enterprise practice, Workers fall into recurring patterns. Choosing the right type for each scenario avoids unnecessary complexity:
1. Timer Worker (Polling)
Runs a task at regular intervals. Ideal for periodic synchronizations, cleanup of expired data and health checks:
| |
2. Queue Worker (Messaging)
Consumes messages from a queue (RabbitMQ, Azure Service Bus, Amazon SQS) in a continuous loop. This is the most common pattern for decoupling services β exactly the scenario described in Database Bottlenecks: Messaging and Pagination:
| |
3. Batch Processing Worker
Processes large volumes of data in paginated batches, avoiding loading everything into memory. It combines naturally with Keyset pagination to process millions of records efficiently:
| |
4. Worker with Parallelism
For scenarios where individual processing is slow (HTTP calls, heavy calculations), the Worker can use controlled parallelism. This connects directly with the concepts in Parallelism in C#: Parallel, PLINQ and Tasks in Practice:
| |
π‘ Tip: Note the
SemaphoreSlim+Task.WhenAllpattern β it enables controlled parallelism without overloading resources. Each item uses its own DI scope (CreateScope), ensuring thatDbContextand other Scoped services are independent across tasks. For more details on controlled parallelism, see Parallelism in C#.
Lifetimes in a Worker Service
This is the most important section of the article β and the one that causes the most bugs in production. DI lifetimes (Singleton, Scoped, Transient) behave differently in Workers compared to Web APIs, because there is no “HTTP request” concept to create scopes automatically.
The Problem: Workers Are Singleton
BackgroundService is registered as a Singleton by AddHostedService<T>(). This means all of its dependencies injected through the constructor also need to be Singleton β or you will have problems:
| |
The Solution: IServiceScopeFactory
The correct pattern is to inject IServiceScopeFactory (which is Singleton) and create scopes manually:
| |
Lifetime Behavior in Workers
| Lifetime | Inject into the Worker constructor? | Via CreateScope? | Behavior |
|---|---|---|---|
| Singleton | β Yes | β Yes | Same instance for the host’s whole life |
| Scoped | β InvalidOperationException | β Yes (required) | New instance per manually created scope |
| Transient | β οΈ Works, but dangerous | β Yes (recommended) | New instance on every resolution β without a scope, it leaks memory |
Why Transient in the Constructor Is Dangerous
| |
β οΈ Warning: In Web APIs, ASP.NET Core creates scopes automatically for each HTTP request. In Workers, you are responsible for creating scopes manually whenever you need Scoped or Transient services. Forgetting to create the scope is the #1 cause of memory leaks and connection pool exhaustion in Workers.
Diagram: DI Scope in Web API vs Worker
In a Web API, each HTTP request automatically creates a DI scope. In the Worker, ExecuteAsync runs in a single Singleton scope β that is why you need to create scopes manually with IServiceScopeFactory.CreateScope() to simulate the same isolation.
Workers in Enterprise Practice
Worker as a Messaging Consumer
The most powerful Worker pattern in enterprise environments is consuming messaging queues. Instead of processing everything synchronously inside the API, the API publishes an event to the queue and the Worker consumes and processes it in the background β exactly the pattern described in Database Bottlenecks: Messaging and Pagination:
| |
Worker for Processing Large Volumes with Pagination
When you need to process millions of records (data migration, balance recalculation, reindexing), the Worker combines with Keyset pagination to process in blocks without overloading memory:
| |
Worker with Async/Await and CancellationToken
The CancellationToken is the Worker’s graceful shutdown contract. When the host receives a stop signal (SIGTERM on Linux, Ctrl+C, docker stop), it fires the token β and the Worker has a limited time to finish what it is doing. This mechanism connects directly with the concepts in Asynchronous Programming with async/await:
| |
| |
βΉοΈ Info: In Docker/Podman containers,
docker stopsendsSIGTERMand waits 10 seconds by default before sendingSIGKILL. If your Worker needs more time for a graceful shutdown, configureShutdownTimeoutin code and--stop-timeouton the container.
Multiple Workers in the Same Host
An application can register several Workers in the same host β each one runs in its own Task:
| |
β οΈ Warning: Workers start in the order in which they were registered, but they run in parallel. Each one’s
StartAsyncis called sequentially (in registration order), but theBackgroundService’sExecuteAsyncruns in the background immediately. If one Worker depends on another already running, useIHostApplicationLifetimefor coordination.
When to Split into Separate Processes vs the Same Host
| Scenario | Same Host | Separate Processes |
|---|---|---|
| Lightweight Workers (timers, health checks) | β Recommended | Unnecessary overhead |
| Workers with different scaling requirements | β | β Scale independently |
| Workers with conflicting dependencies | β | β Dependency isolation |
| Workers consuming different queues | Depends on the volume | β Scale per queue |
| Deploy on Kubernetes | β | β One Deployment per Worker |
| Deploy on VPS/VM | β (systemd services) | Complex management |
Running Workers as an Operating System Service
In production, Workers run as services managed by the operating system:
Linux β systemd
| |
Docker / Podman
| |
| |
Tips and Good Practices
Always use
IServiceScopeFactoryfor Scoped services β never injectDbContext, repositories or Scoped services directly into the Worker’s constructor. Create scopes manually inside the loop.Propagate the
CancellationTokento all async operations β fromExecuteAsyncall the way to the database call, the messaging queue and theHttpClient. A Worker that ignores the token takes a long time to stop and can lose data on shutdown.Handle exceptions inside the loop β in .NET 8, unhandled exceptions in
ExecuteAsyncstop the host. Use try/catch with retry and logging to prevent a transient error from killing the Worker.Configure
HostOptions.ShutdownTimeoutappropriately β the 30-second default may be insufficient for Workers that process large batches. Align it with Docker/Podman’s--stop-timeout.Use
IHttpClientFactoryinstead ofnew HttpClient()β in Singleton services such as Workers, instantiatingHttpClientdirectly causes socket leaks and stale DNS.IHttpClientFactorymanages the handler pool.Prefer Keyset pagination for batch processing β avoid
Skip/Takewith OFFSET for large datasets. Keyset (WHERE Id > lastId) keeps performance constant regardless of volume.Split Workers by responsibility β a Worker should do one thing well. “Swiss army knife” Workers with multiple responsibilities are hard to scale, monitor and debug.
Add health checks to the Worker β use
IHealthCheckand expose a minimal HTTP endpoint (withMapHealthChecks) so that orchestrators (Kubernetes, Docker) can check whether the Worker is healthy.
Conclusion
.NET Workers and Background Services are .NET’s native solution for background processing β no hacks with Task.Run() inside controllers, no third-party libraries for scenarios the framework already handles. The Generic Host provides the same configuration, DI and logging infrastructure as Web APIs, allowing teams to share patterns and knowledge between HTTP projects and Workers.
The most critical point β and the one that generates the most bugs in production β is understanding that Workers are Singleton. This fundamentally changes how you deal with DbContext, repositories and any Scoped service. The IServiceScopeFactory.CreateScope() pattern is not optional: it is the foundation of every Worker that accesses a database or stateful services.
For high-volume enterprise applications, combining Workers with messaging for decoupling, Keyset pagination for batch processing, controlled parallelism with SemaphoreSlim and async/await with CancellationToken forms a complete arsenal for building systems that process millions of records reliably, without locking up the API and without losing data on shutdown.
Read Also
- Database Bottlenecks: Messaging and Pagination β how Workers consume messaging queues to relieve the database in high-volume scenarios.
- Pagination in REST APIs with C# and EF Core 8 β Keyset and Offset pagination for batch processing inside Workers.
- Parallelism in C#: Parallel, PLINQ and Tasks in Practice β how to combine
Task.WhenAll,SemaphoreSlimandParallel.ForEachAsyncinside Workers for controlled parallelism. - Asynchronous Programming in C#: async/await and Threads β async/await and CancellationToken fundamentals that are the foundation of every Worker Service.
- .NET 8+ Configuration Pipeline: IOptions, Secrets and Docker β the configuration pipeline shared between Workers and Web APIs, including Docker secrets with
AddKeyPerFile. - EF Core Migrations: Multi-Project, Secrets and Scaffolding β managing migrations and secrets in multi-layer projects with Workers.
References
- Worker Services in .NET β Microsoft Learn β official documentation for Workers and Background Services in .NET, including examples and good practices.
- BackgroundService Class β Microsoft Learn β
BackgroundServiceAPI reference, includingExecuteAsyncandStopAsync. - IHostedService Interface β Microsoft Learn β reference for the base interface for hosted services with
StartAsyncandStopAsync. - .NET Generic Host β Microsoft Learn β how the Generic Host configures DI, configuration and logging for Workers and Web APIs.
- Dependency Injection Lifetimes β Microsoft Learn β official documentation for the Singleton, Scoped and Transient lifetimes in .NET.
- Use scoped services within a BackgroundService β Microsoft Learn β official guide for using
IServiceScopeFactoryin Background Services. - RabbitMQ .NET Client β RabbitMQ Docs β official documentation for the RabbitMQ .NET client, including
AsyncEventingBasicConsumer. - Create Windows Services using BackgroundService β Microsoft Learn β how to run Workers as Windows Services with
UseWindowsService(). - BackgroundServiceExceptionBehavior β Microsoft Learn β configuring the behavior for unhandled exceptions in
ExecuteAsyncstarting with .NET 8. - blog-zocateli-sample repository β Workers β Full source code for the examples in this article

By commenting, you agree to our Privacy Policy, Terms of Use, Data Deletion Policy.