Make BackgroundService startup not block host startup

[vova-lantsov-dev]

Background and motivation

IHostedService.StartAsync executes inline (and sequentially) when IHost.Start/StartAsync is called. With a BackgroundService, this causes lots of confusion since running code that isn't async in ExecuteAsync or StartAsync also runs inline. The workaround today is to call Task.Run on your own code or to use Task.Yield(), but this is not obvious.

API Proposal

public class BackgroundService : IHostedService
{
+    public virtual bool StartAsynchronously { get; } = true;
}

NOTE: StartAsynchronously would be true by default.

API Usage

class MyBackgroundService : BackgroundService
{
    // It's true by default, this is just for illustration.
    public override bool StartAsynchronously => true;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(5));

        while (await timer.WaitForNextTickAsync(stoppingToken))
        {
            Console.WriteLine("Hello from background service!");
        }
    }
}

Alternative Designs

An optional constructor argument.

Original Issue

BackgroundService blocked the execution of whole host

Describe the bug

When I run the specific foreach cycle in BackgroundService-derived class, it blocks the whole host from starting. Even second hosted service doesn't start. When I comment foreach cycle, everything works as expected.

To Reproduce

TargetFramework: netcoreapp2.1
Version: 2.1.12

Use following hosted service: NotificationRunner.cs
And singleton: NotificationService.cs

Expected behavior

BackgroundService.ExecuteAsync should work in background without blocking even if it has blocking code. As you can see in NotificationService class, cancellation of enumerator is based on IApplicationLifetime.ApplicationStopping, but anyway it shouldn't affect the host startup because BackgroundService is expected to run in background :)

Screenshots

When foreach cycle exists

Then execution is blocked on this cycle

But when foreach cycle is commented

Then execution continues as expected

(But why twice?)

Additional context

.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview7-012821
 Commit:    6348f1068a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.0.100-preview7-012821\

Host (useful for support):
  Version: 3.0.0-preview7-27912-14
  Commit:  4da6ee6450

.NET Core SDKs installed:
  2.1.700 [C:\Program Files\dotnet\sdk]
  2.1.701 [C:\Program Files\dotnet\sdk]
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.2.300 [C:\Program Files\dotnet\sdk]
  2.2.301 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  3.0.100-preview7-012821 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview7.19365.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview7-27912-14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0-preview7-27912-14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

[vova-lantsov-dev]

Fixed by adding await Task.Yield() at the top of method

[dcarr42]

Yeah the method shouldn't be blocking. Wrapping in a Task.Run will also dispatch.

[rynowak]

Reopening this so we can look at it. I'm glad that you solved your problem but we should try to solve this so others don't hit it.

[rynowak]

@anurse @Tratcher - thoughts?

It looks like if your run a bunch of CPU-bound code in ExecuteAsync then it will stall startup until that work reaches the first actual async point.

See: https://github.com/aspnet/Extensions/blob/master/src/Hosting/Abstractions/src/BackgroundService.cs#L33

In health checks I ended up inserting a Task.Yield to avoid this. We'd get an additional state machine in this case, 1 per service, but it seems like a worthwhile tradeoff.

[vova-lantsov-dev]

Async method executes synchronously till first await is reached (it means BackgroundService.ExecuteAsync won't return till await is called). So this problem is not related to ASP.NET Core, but to async compilation.

[vova-lantsov-dev]

But it would be great if you can solve this case

[Tratcher]

It's a trade off. There may be reasonable initialization code the service needs to run before letting the app continue. It's within the service's control how long it blocks.

[kostya9]

I have also encountered this issue and solved it in the same way as @vova-lantsov-dev did. However, my main concern is that this behavior is not straightforward, would be nice if this was more predictable or documented somewhere

[analogrelay]

I think it's reasonable to dispatch ExecuteAsync to the thread-pool in StartAsync. If the user wants to perform initialization that must complete before letting the app continue, they can override StartAsync itself, right?

public class MyService: BackgroundService
{
	public override async Task StartAsync()
	{
		await InitializeAsync();
		await base.StartAsync();
	}

	public override async Task ExecuteAsync()
	{
		// Do background stuff...
	}
}

It's called BackgroundService. It seems odd that you can block the foreground in the default case. If we're not happy with overriding StartAsync we could add a new InitializeAsync that is called and awaited during StartAsync.

This would be a breaking change however, since existing services may depend on the initial synchronous work blocking the startup process.

[davidfowl]

Sorry I never did this change but enough people have hit it now I think we should just do it. https://github.com/aspnet/Extensions/tree/davidfowl/background-service

[pholly]

I just ran into this and was about to leave feedback on the docs. Glad I checked here first. The Worker template works because it awaits a Task.Delay. Change that to a Thread.Sleep, remove async keyword, and return a CompletedTask from ExecuteAsync and it can be easily reproduced. Execution is not returned to the caller so StartAsync never finishes and the host never finishes initialization so cancellationtoken does not work and any other HostedServices registered would never start.

[pholly]

@davidfowl I just looked at the BackgroundService changes in your branch - would it make more sense for the Host to Task.Run each hosted service instead of depending on the hosted service to do it?