Add the Context notion. (#424)

* Initial context integration.

* Update api-propagators.md

* Remove the mention of binary propagators.

* Mention that the tracing's propagation must happen internally.

* Make context -> Context.

* Rename api-context.md to context.md

* Remove the notion of key.

* Add a note on Context being a SDK API.

* Note on the optional global operations.

* Clean up pass.

* Remove context-prop from the *tracing* index.

* Update specification/api-propagators.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/context.md

Co-Authored-By: Christian Neumüller <[email protected]>

* First clean up pass over Context.

* Update specification/api-propagators.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Overview and layout updates.

* Make the MD linter happy (long lines though).

* Update specification/context.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/context.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/context.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/context.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Use MUST/SHOULD for clarifying propagators expectations.

* Move the IsRemote deserialization note to the Extraction section.

* Improve wording for Context handling in Extraction.

* Clarify the protected global methods in context.md

* Remove the extra explanation of Context in overview.md

* Improve the wording for set/remove values in context.md

* Update specification/api-propagators.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: Christian Neumüller <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: Tyler Yahn <[email protected]>

* Fix style?

* Clarify the Context usage in Extract.

* Update specification/context.md

Co-Authored-By: Tyler Yahn <[email protected]>

* Clarify IsRemote on Extract.

* More style changes.

* OTel MUST provide Context impl if none exists.

* Update specification/context.md

Co-Authored-By: Tyler Yahn <[email protected]>

* Update specification/context.md

Co-Authored-By: Tyler Yahn <[email protected]>

* Update specification/context.md

Co-Authored-By: Tyler Yahn <[email protected]>

* Reword the optionality of Context in propagators.

* Clarify again the expected usage for implicit `Context`.

* Update specification/context.md

Co-Authored-By: Tyler Yahn <[email protected]>

* Be more strict on the Context parameters.

* Clarify that SDK/OTel instrumentation only SHOULD use global Context.

* Mention that Context will be used through concerns APIs.

* Update specification/api-propagators.md

Co-Authored-By: Tyler Yahn <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: Tyler Yahn <[email protected]>

* We do not use the Required word anymore.

* Do not require extract to set null/empty upon errors.

* Better wording for failed extraction values.

* Update specification/context.md

Co-Authored-By: Yuri Shkuro <[email protected]>

* Clarify ONCE that Context is immutable.

* Use "derived" instead of "child" for Context-Context relationships.

* Remove a misleading line.

* Mention tracing uses Context specifially.

* Update specification/api-propagators.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: John Watson <[email protected]>

* Clarify that Context keeps the untouched original values.

* Clarify a new Context is created only for write operations.

* Misc fixes.

* Set current Context should return a handle.

* Note on global Context operations.

* Remove the Remove operation on Context.

* Clarify the usage of cross-cutting concern APIs.

* Restore "Create a key" section.

* Minor fix to a previous commit.

* Add a detach operation.

* Update specification/api-propagators.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/api-propagators.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/context.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/context.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/context.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/context.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/context.md

Co-Authored-By: John Watson <[email protected]>

* Update specification/api-tracing.md

Co-Authored-By: John Watson <[email protected]>

* Use value instead of object for Token.

Co-authored-by: Christian Neumüller <[email protected]>
Co-authored-by: Tyler Yahn <[email protected]>
Co-authored-by: Yuri Shkuro <[email protected]>
Co-authored-by: John Watson <[email protected]>
Co-authored-by: Bogdan Drutu <[email protected]>

Author: 6 people

specification/api-propagators.md

@@ -26,16 +26,17 @@ Cross-cutting concerns send their state to the next process using
 context data to and from messages exchanged by the applications.
 Each concern creates a set of `Propagator`s for every supported `Format`.
 
+Propagators leverage the `Context` to inject and extract data for each
+cross-cutting concern, such as traces and correlation context.
+
 The Propagators API is expected to be leveraged by users writing
 instrumentation libraries.
 
-Propagators API currently consists of one `Format`:
+The Propagators API currently consists of one `Format`:
 
-- `HTTPTextFormat` is used to inject and extract a value as text into carriers that travel
+- `HTTPTextFormat` is used to inject values into and extract values from carriers as text that travel
   in-band across process boundaries.
 
-Deserializing must set `IsRemote` to true on the returned `SpanContext`.
-
 A binary `Format` will be added in the future.
 
 ## HTTP Text Format
@@ -75,7 +76,7 @@ Injects the value downstream. For example, as http headers.
 
 Required arguments:
 
-- the cross-cutting concern value to be injected, such as `SpanContext` or `DistributedContext`.
+- A `Context`. The Propagator MUST retrieve the appropriate value from the `Context` first, such as `SpanContext`, `CorrelationContext` or another cross-cutting concern context. For languages supporting current `Context` state, this argument is OPTIONAL, defaulting to the current `Context` instance.
 - the carrier that holds propagation fields. For example, an outgoing message or http request.
 - the `Setter` invoked for each propagation key to add or remove.
 
@@ -101,18 +102,23 @@ The implemenation SHOULD preserve casing (e.g. it should not transform `Content-
 
 ### Extract
 
-Extracts the value from upstream. For example, as http headers.
+Extracts the value from an incoming request. For example, from the headers of an HTTP request.
 
-If the value could not be parsed, the underlying implementation will decide to return an
-object representing either an empty value, an invalid value, or a valid value. Implementations
-MUST not return null.
+If a value can not be parsed from the carrier for a cross-cutting concern,
+the implementation MUST NOT throw an exception. It MUST store a value in the `Context`
+that the implementation can recognize as a null or empty value.
 
 Required arguments:
 
+- A `Context`. For languages supporting current `Context` state this argument is OPTIONAL, defaulting to the current `Context` instance.
 - the carrier holds propagation fields. For example, an outgoing message or http request.
 - the instance of `Getter` invoked for each propagation key to get.
 
-Returns the non-null cross-cutting concern extracted value.
+Returns a new `Context` derived from the `Context` passed as argument,
+containing the extracted value, which can be a `SpanContext`,
+`CorrelationContext` or another cross-cutting concern context.
+
+If the extracted value is a `SpanContext`, its `IsRemote` property MUST be set to true.
 
 #### Getter argument
 

specification/api-tracing.md

@@ -122,13 +122,15 @@ mechanism, for instance the `ServiceLoader` class in Java.
 
 The `Tracer` MUST provide functions to:
 
-- Get the currently active `Span`
 - Create a new `Span`
+
+The `Tracer` SHOULD provide methods to:
+
+- Get the currently active `Span`
 - Make a given `Span` as active
 
-The `Tracer` SHOULD allow end users to configure other tracing components that
-control how `Span`s are passed across process boundaries, including the text
-format `Propagator` used to serialize `Span`s created by the `Tracer`.
+The `Tracer` MUST internally leverage the `Context` in order to get and set the
+current `Span` state and how `Span`s are passed across process boundaries.
 
 When getting the current span, the `Tracer` MUST return a placeholder `Span`
 with an invalid `SpanContext` if there is no currently active `Span`.
@@ -139,18 +141,13 @@ SHOULD create each new `Span` as a child of its active `Span` unless an
 explicit parent is provided or the option to create a span without a parent is
 selected, or the current active `Span` is invalid.
 
-The `Tracer` MUST provide a way to update its active `Span`, and MAY provide
+The `Tracer` SHOULD provide a way to update its active `Span` and MAY provide
 convenience functions to manage a `Span`'s lifetime and the scope in which a
 `Span` is active. When an active `Span` is made inactive, the previously-active
 `Span` SHOULD be made active. A `Span` maybe finished (i.e. have a non-null end
 time) but stil active. A `Span` may be active on one thread after it has been
 made inactive on another.
 
-The implementation MUST provide a text `Propagator`, which the
-`Tracer` SHOULD use by default if other propagators are not configured. SDKs
-SHOULD use the W3C HTTP Trace Context as the default text format. For more
-details, see [trace-context](https://github.com/w3c/trace-context).
-
 ## SpanContext
 
 A `SpanContext` represents the portion of a `Span` which must be serialized and

specification/context.md

@@ -0,0 +1,111 @@
+# Context
+
+<details>
+<summary>
+Table of Contents
+</summary>
+
+- [Overview](#overview)
+- [Create a key](#create-a-key)
+- [Get value](#get-value)
+- [Set value](#set-value)
+- [Optional operations](#optional-operations)
+    - [Get current Context](#get-current-context)
+    - [Attach Context](#attach-context)
+    - [Detach Context](#detach-context)
+
+</details>
+
+## Overview
+
+A `Context` is a propagation mechanism which carries execution-scoped values
+across API boundaries and between logically associated execution units.
+Cross-cutting concerns access their data in-process using the same shared
+`Context` object.
+
+A `Context` MUST be immutable, and its write operations MUST
+result in the creation of a new `Context` containing the original
+values and the specified values updated.
+
+Languages are expected to use the single, widely used `Context` implementation
+if one exists for them. In the cases where an extremely clear, pre-existing
+option is not available, OpenTelemetry MUST provide its own `Context`
+implementation. Depending on the language, its usage may be either explicit
+or implicit.
+
+Users writing instrumentation in languages that use `Context` implicitly are
+discouraged from using the `Context` API directly. In those cases, users will
+manipulate `Context` through cross-cutting concerns APIs instead, in order to
+perform operations such as setting trace or correlation context values for
+a specified `Context`.
+
+A `Context` is expected to have the following operations, with their
+respective language differences:
+
+## Create a key
+
+Keys are used to allow cross-cutting concerns to control access to their local state,
+and they cannot be guessed by third parties. It is recommended that concerns mediate
+data access via an API, rather than provide direct public access to their keys.
+
+The API MUST accept the following parameter:
+
+- The key identifier. Different languages may impose different restrictions on the expected types, so this parameter remains an implementation detail.
+
+The API MUST return an opaque object representing the newly created key.
+
+## Get value
+
+Concerns can access their local state in the current execution state
+represented by a `Context`.
+
+The API MUST accept the following parameters:
+
+- The `Context`.
+- The key.
+
+The API MUST return the value in the `Context` for the specified key.
+
+## Set value
+
+Concerns can record their local state in the current execution state
+represented by a `Context`.
+
+The API MUST accept the following parameters:
+
+- The `Context`.
+- The key.
+- The value to be set.
+
+The API MUST return a new `Context` containing the new value.
+
+## Optional Global operations
+
+These operations are expected to only be implemented by languages
+using `Context` implicitly, and thus are optional. These operations
+SHOULD only be used to implement automatic scope switching and define
+higher level APIs by SDK components and OpenTelemetry instrumentation libraries.
+
+### Get current Context
+
+The API MUST return the `Context` associated with the caller's current execution unit.
+
+### Attach Context
+
+Associates a `Context` with the caller's current execution unit.
+
+The API MUST accept the following parameters:
+
+- The `Context`.
+
+The API MUST return a value that can be used as a `Token` to restore the previous
+`Context`.
+
+### Detach Context
+
+Resets the `Context` associated with the caller's current execution unit
+to the value it had before attaching a specified `Context`.
+
+The API MUST accept the following parameters:
+
+- A `Token` that was returned by a previous call to attach a `Context`.

specification/overview.md

@@ -235,10 +235,18 @@ OpenTelemetry
 [proto](https://github.com/open-telemetry/opentelemetry-proto/blob/a46c815aa5e85a52deb6cb35b8bc182fb3ca86a0/src/opentelemetry/proto/agent/common/v1/common.proto#L28-L96)
 for an example.
 
+## Context Propagation
+
+All of OpenTelemetry cross-cutting concerns, such as traces and metrics,
+share an underlying `Context` mechanism for storing state and
+accessing data across the lifespan of a distributed transaction.
+
+See the [Context](context.md)
+
 ## Propagators
 
 OpenTelemetry uses `Propagators` to serialize and deserialize cross-cutting concern values
-such as  `SpanContext` and `DistributedContext` into a `Format`. Currently there is one
+such as `SpanContext` and `CorrelationContext` into a `Format`. Currently there is one
 type of propagator:
 
 - `HTTPTextFormat` which is used to inject and extract a value as text into carriers that travel