[Feature] Add host integration with (Azure) Durable Task Scheduler

[philliphoff]

Description

Durable Task Scheduler (DTS) is a new Azure service (currently in public preview) for applications built on Durable Task, both as part of Azure Durable Functions as well as stand-alone compute. Developers with DTS dependencies may wish to orchestrate the startup of and/or connection strings related to DTS instances as part of their overall .NET Aspire application. In particular, orchestrating the startup of a DTS emulator instance for local development.

Proposed Solution

Add a CommunityToolkit.Aspire.Hosting.DurableTask host integration library. This library would enable use of DTS (via emulator) as shown:

var builder = DistributedApplication.CreateBuilder(args);

var scheduler = builder.AddDurableTaskScheduler("scheduler")
                       .RunAsEmulator();

var taskHub = scheduler.AddDurableTaskHub("taskhub");

builder.AddProject<Projects.MyApp>("myapp")
       .WithReference(taskHub);

builder.Build().Run();

The scheduler and task hub resources will generate and propagate appropriate connection strings, as well as expose properties for customization (e.g. scheduler and task hubs with names that differ from their respective resource names, authentication mechanism, emulator options, etc). When using the emulator, commands will be automatically added to the resources so developers can easily open the DTS dashboard from the .NET Aspire dashboard.

For existing (Azure) DTS instances, connection strings can be passed through parameters:

var builder = DistributedApplication.CreateBuilder(args);

var scheduler =
    builder.AddDurableTaskScheduler("scheduler")
           .RunAsExisting(builder.AddParameter("scheduler-connection-string"));

var taskHub =
    scheduler.AddTaskHub("taskhub")
             .WithTaskHubName(builder.AddParameter("taskhub-name"));

builder.AddProject<Projects.MyApp>("myapp")
       .WithReference(taskHub);

builder.Build().Run();

Note: deployment of DTS instances based on the application model is a PR for another day.

Existing Work

This integration work has been started here:
https://github.com/philliphoff/CommunityToolkit.Aspire/tree/philliphoff-dts-integration

• Host project added
• Samples added
• Unit tests added
• Integration tests added
• Tests added to the CI pipeline
• Community Toolkit docs updated
• .NET Aspire docs updated
• NuGet Packaging properties added

Related

microsoft/aspire#8926

[aaronpowell]

Would it be fair to assume that there are three integrations that are needed here, one for hosting, one for hosting/deployment in Azure, and one for the client?

[philliphoff]

Would it be fair to assume that there are three integrations that are needed here, one for hosting, one for hosting/deployment in Azure, and one for the client?

I think so, yes, though I've not had the time to look into the details of deployment or client integration. I know deployment has changed significantly since my original Dapr implementation. Client integration may be complicated by the fact that Durable features are exposed in a number of places (e.g. Azure Functions plus two different SDKs, which are themselves split into separate "client" and "worker" libraries), which potentially means quite a few new assemblies to cover all use cases.

[aaronpowell]

Based on the discussion in the Aspire repo, it looks like that is a better home for it, so I'm going to close there one here.

[cecilphillip]

@philliphoff did you end up publishing this somewhere else?

[philliphoff]

@cecilphillip It made it as far as a PR in the main Aspire repo but after a lot of back and forth--there was a lot of API churn at the time--it kind of got put back on the shelf in favor of other things. My original implementation was probably too much at the time; I have a smaller, more targeted addition that I've been trying to clean up and get ready for a PR as I find the time.