TC39 AsyncContext

[legendecas]

Request for Node.js Position on an Emerging Specification

• Specification title: AsyncContext
• Specification or proposal URL (if available): https://github.com/tc39/proposal-async-context/
• Explainer URL (if available): (includes potential future extensions)
  • Web integration: https://github.com/tc39/proposal-async-context/blob/master/WEB-INTEGRATION.md
  • Mutation explainer: https://github.com/tc39/proposal-async-context/blob/master/MUTATION-SCOPE.md
  • Continuation flow: https://github.com/tc39/proposal-async-context/blob/master/CONTINUATION.md
• Proposal author(s) (@-mention GitHub accounts):
  • @andreubotella
  • @legendecas
  • @jridgewell
• Node.js collaborators or ecosystem members who can provide input (optional):
  • @nodejs/diagnostics

Notes

The AsyncContext proposal adds an API that shares similar scopes with Node.js stable API AsyncLocalStorage into the ECMAScript. Moreover, AsyncContext adds generator/yield support and making the semantics of future using declaration support more intuitive in generator function bodies.

The proposal will expand the usages of Node.js stable API AsyncLocalStorage to the Web and other JS runtimes.

[mcollina]

I reviewed this extensively, and we should hold a session at the collaborator summit to get a shared position from the Node.js project.

The spec is divided into multiple parts, with the mutation and continuation flow descriptions left for future work. I’ll let @Qard expand on the need for continuation flow while I focus on the mutation scope.

While I think this proposal is a good first step, I believe that without the mutation scope addition, it would not replace AsyncLocalStorage or the "old" async_hooks. Let me explain why.

The ecosystem is massively using .enterWith() and equivalent techniques. It’s an _unsafe_ escape hatch we left open that would allow users to “patch” the current context synchronously because, in quite a few relevant cases, it’s not possible to wrap code in another function, like what using .run() would require. enterWith() isn’t without problems either, see nodejs/node#53037.

Here is a list of modules using synchronous context access (I found them using github search):

• cls-tracer with 64k weekly downloads
• cls-hooked with almost 2 million weekly downloads
• dd-trace with 3 million weekly downloads
• nest-cls with 246k weekly downloads
• @sveltejs/kit with 513k weekly downloads which uses enterWith in its dev mode
• @mikro-orm/core with 254k weekly downloads

Both Deno and Bun also implement enterWith() because it is so commonly used. Both initially tried to skip it as it’s marked experimental but many users requested it be added.

Note that they are using this API because no alternative exists, or the alternatives would require API changes in other ecosystem modules, generically lowering the DX in some cases.

The current spec does not address these cases, while the “future extensions” do: either “mutation scope” or “continuation flow” provide enough escape hatch to implement this across the Node.js ecosystem.

[Qard]

@mcollina Minor correction: Bun did have enterWith(...) from the start, but it didn't work correctly. They had to fix it to support Next.js Pages Router. Deno omitted it initially, but added it later to get SvelteKit to work.

---

The core purpose of asynchronous context is to behave like a global variable but anchored to a point descending from the application root context where continuing execution is unique to that particular branch of execution. It is typically associated with an http request, but can be tied to any sub-tree of the async call graph.

It is generally the user's expectation that once a value is set to the context it should be available to all logically following execution within that async execution context until explicitly changed or deleted. This mirrors the behaviour of if they stored the data in a global within an app which only handled a single concurrent request. It doesn’t matter if the execution is descending into calls or flowing back out, a global should still behave like a global. What matters is not where the variable changes but where the variable was first used within the async call graph–something like where a global was defined, but existing async context APIs don’t provide any distinction between initial definition (which defines scope binding) and mutation.

I have been told before that one can “just store a map”, but that approach is insufficient as it does not provide any programmatic awareness of the point at which this global-like thing should be considered “defined” from the flow perspective, and it’s typically not even a singular place for all pieces of data.

Tracing products, for example, often will have a separate context scope for each span around every async call, of which there are many, and then a request-level scope to aggregate span contexts into either internal to the process or (more optimally) externally. This involves many different ranges of context flow depending on the piece of data, which is impossible to express with a map which would be rooted to only the one spot where the map was constructed and stored.

Datadog presently works around this by destructuring and reconstructing maps potentially thousands of times per request, which is quite costly, but there's no alternative with how async context works presently.

For a singular context value which you control fully end-to-end a map may be workable, but the vast majority of async context use cases involve sharing data between disparate systems where there is not one clear owner or user of the store data and very often stores need to be able to propagate multiple different pieces of data flowing from different points in the async call graph. This is why AsyncLocalStorage was made multi-tenant in the first place and is a necessary feature for context management to be useful in the majority of scenarios.

In large products, it is very common for multiple teams to exist in the same codebase and it’s often difficult to coordinate passing data through code from other teams. It’s very common for teams to use AsyncLocalStorage in this sort of scenario, and it mostly works…until it doesn’t. Then they come complaining that it doesn’t work the way they thought it should, which is behaving as if it was a global only accessible for that request, and flowing temporally like modifying a global normally would.

const app = someRouter()
app.store = new SomeContextStore()

app.use(async function myTeamMiddleware(req, res, next) {
  app.store.set('second-middleware-called', false)
  await next()

  // This doesn't work unless context flows forward rather than being
  // terminated by awaits discarding the context and restoring the state
  // from before the await.
  if (app.store.get('second-middleware-called')) {
    // We know the other team delegated to our other middleware
  }
})

// Other team provides a middleware. It is a black box to us,
// but we must be able to pass data through it to our other
// middleware and back to our first middleware when the stack
// reaches that point.
app.use(async function otherTeamMiddleware(req, res, next) {
  // Other team does some async stuff...
  if (delegateToOtherTeamThing) {
    await next()
  }
})

app.use(async function myTeamSecondMiddleware(req, res, next) {
  app.store.set('second-middleware-called', true)
  await next()
})

This basic scenario is impossible with the current design of AsyncContext and challenging to do with AsyncLocalStorage as it requires some careful use of enterWith(...) and a deep understanding of the execution scheduling of Node.js to get it right.

The other problematic pattern I see often is people thinking that a map will work, or assuming that something which looks too much like a map behaves like a map.

// node_modules/some-app-router/config.js
export default class ConfigStore {
  #store = new AsyncLocalStorage()

  set(key, value) {
    const current = this.#current
    this.#store.enterWith({ ...current, [key]: value })
  }

  get(key) {
    return this.#current[key]
  }

  get #current() {
    return this.#store.getStore() ?? {}
  }
}

// node_modules/some-app-router/app.js
import ConfigStore from './config.js'

export default function app() {
  return {
    config: new ConfigStore(),
    // ... other stuff for routing
  }
}

// config/database.js
// User doesn't necessarily know if this is called per-request or once at startup
export default function plugin(app) {
  // User doesn't know this is backed by AsyncLocalStorage, they just see a Map-like thing.
  app.config.set('database', new Database())
}

// server.js
import makeApp from 'some-app-router'
import plugin from './config/database.js'

export default const app = makeApp()

// What is the app doing with the plugin function? Calling it right away? Per-request? Who knows?
app.plugin(plugin)

app.use((req, res, next) => {
  // Is the database available? No way to know!
  const user = await app.config.get('database').getUserFromSession(req)

  // The framework intended for this to work though, which is why it uses AsyncLocalStorage internally
  app.config.set('user', user)
})

This forward flow has been described as a leak, which it certainly is not as it will only propagate if execution continues to propagate, which is the intended behaviour. It is no more a leak than a Map is a leak because it has the potential to be used to store data indefinitely. If that is the intended behaviour then it’s not a leak. It’s also incapable of growing in reference containment unless execution flow grows in branches without terminating, in which case you’re already leaking a whole lot more than just the variable, and everything would suffer the same fate, including plain lexically accessible variables.

It has also been called a security issue because it enables systems to communicate indirectly, but that’s the point of the system and that capability is gated through the context variable instance so is in no way a security issue either as you’re already intentionally sharing the variable needed for such access.

---

A better approach to context flow is to decouple the providing of scopes from the mutation of context state by using a copy-on-write slot to link context propagation flows. Whenever the store value is set it copies the set of pointers for all stores at that point and swaps the pointer contained in the context slot for that store instance in the new pointer set. When an async task is constructed it copies whatever pointer set is contained at that time. By working this way, tasks which are constructed before a change don't have their context value swapped out from under them because they've already copied whatever the pointer was at the time of their construction.

The following is an example of the semantics which would better suit the flows which most users expect.

function syncCall() {
  assert.strictEqual(store.get(), 'before syncCall')
  store.set('from syncCall')
}

async function asyncCall() {
  assert.strictEqual(store.get(), 'before asyncCall')
  store.set('should not be visible')
  await delay(100)
  store.set('from asyncCall')
}

const store = new AsyncContext()

const promise = store.scope(async () => {
  // Context should flow through sync calls as if the data was stored in a global
  store.set('before syncCall')
  syncCall()
  assert.strictEqual(store.get(), 'from syncCall')

  // Context should also flow through async calls similarly, as if a global existed
  // only within this scope.
  store.set('before asyncCall')
  const promise = asyncCall()
  assert.strictEqual(store.get(), 'before asyncCall')
  await promise
  assert.strictEqual(store.get(), 'from asyncCall')

  // Should only escape scope wrappers asynchronously
  store.set('done scope')
})

// Does not flow any context out of the scope synchronously.
assert.strictEqual(myStore.get(), undefined)

// It does flow out asynchronously though as execution internal to the scope now
// becomes causally responsible for the remaining execution of this scope.
await promise
assert.strictEqual(myStore.get(), 'done scope')

// Sync scopes terminate at the end of their window
store.scope(() => {
  store.set('should not be visible')
})
assert.strictEqual(store.get(), 'done scope')

One of the most important benefits of splitting scope definition from context mutation is that you gain the ability to provide scopes implicitly. For example, in the example above you may have noticed that context mutations don't escape async functions, even when made in the initial synchronous segment of the asynchronous function. This behaviour is achievable by async functions implicitly providing a scope around themselves. This would allow for applications to just plainly rely on set/get calls without need for scope nesting which doesn't play nice with the semantic flattening that async/await provides, allowing for good ergonomics while also behaving in a much more consistent manner.

The fact that async functions have a synchronous segment at the start, while sensible from a performance perspective, is a source of much user confusion from a side-effects perspective. By containing that Zalgo-esque irregularity you get code which behaves in a very clear and sensible way while still having the technical benefit of a sync function prelude. The flow semantics also become a lot more clear that sync scopes box everything within themselves, but async continuations flow context out to any continuing execution which can raise contexts upward, but only in scenarios where a user would have expected that to have the semantics of a global variable.