Merge pull request #3818 from handrews/rfc6570-304

Appendix on RFC6570-derived behavior + `allowReserved` (3.0.4)

Author: handrews

versions/3.0.4.md

@@ -1073,11 +1073,13 @@ Field Name | Type | Description
 ---|:---:|---
 <a name="parameterStyle"></a>style | `string` | Describes how the parameter value will be serialized depending on the type of the parameter value. Default values (based on value of `in`): for `query` - `form`; for `path` - `simple`; for `header` - `simple`; for `cookie` - `form`.
 <a name="parameterExplode"></a>explode | `boolean` | When this is true, parameter values of type `array` or `object` generate separate parameters for each value of the array or key-value pair of the map. For other types of parameters this property has no effect. When [`style`](#parameterStyle) is `form`, the default value is `true`. For all other styles, the default value is `false`.
-<a name="parameterAllowReserved"></a>allowReserved | `boolean` | Determines whether the parameter value SHOULD allow reserved characters, as defined by [RFC3986](https://tools.ietf.org/html/rfc3986#section-2.2) `:/?#[]@!$&'()*+,;=` to be included without percent-encoding. This property only applies to parameters with an `in` value of `query`. The default value is `false`.
+<a name="parameterAllowReserved"></a>allowReserved | `boolean` | When this is true, parameter values are serialized  using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are [not allowed in the query string](https://datatracker.ietf.org/doc/html/rfc3986#autoid-24) (`[`, `]`, `#`), or have a special meaning in `application/x-www-form-urlencoded` (`-`, `&`, `+`); see Appendices [C](#usingRFC6570Implementations) and [E](#percentEncodingAndFormMediaTypes) for details. This property only applies to parameters with an `in` value of `query`. The default value is `false`.
 <a name="parameterSchema"></a>schema | [Schema Object](#schemaObject) \| [Reference Object](#referenceObject) | The schema defining the type used for the parameter.
 <a name="parameterExample"></a>example | Any | Example of the parameter's potential value; see [Working With Examples](#working-with-examples).
 <a name="parameterExamples"></a>examples | Map[ `string`, [Example Object](#exampleObject) \| [Reference Object](#referenceObject)] | Examples of the parameter's potential value; see [Working With Examples](#working-with-examples).
 
+See also [Appendix C: Using RFC6570 Implementations](#usingRFC6570Implementations) for additional guidance.
+
 ###### Fixed Fields and considerations for use with `content`
 
 For more complex scenarios, the [`content`](#parameterContent) property can define the media type and schema of the parameter, as well as give examples of its use.
@@ -1624,10 +1626,12 @@ Field Name | Type | Description
 <a name="encodingHeaders"></a>headers | Map[`string`, [Header Object](#headerObject) \| [Reference Object](#referenceObject)] | A map allowing additional information to be provided as headers, for example `Content-Disposition`.  `Content-Type` is described separately and SHALL be ignored in this section. This property SHALL be ignored if the request body media type is not a `multipart`.
 <a name="encodingStyle"></a>style | `string` | Describes how a specific property value will be serialized depending on its type.  See [Parameter Object](#parameterObject) for details on the [`style`](#parameterStyle) property. The behavior follows the same values as `query` parameters, including default values. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`.
 <a name="encodingExplode"></a>explode | `boolean` | When this is true, property values of type `array` or `object` generate separate parameters for each value of the array, or key-value-pair of the map.  For other types of properties this property has no effect. When [`style`](#encodingStyle) is `form`, the default value is `true`. For all other styles, the default value is `false`. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`.
-<a name="encodingAllowReserved"></a>allowReserved | `boolean` | Determines whether the parameter value SHOULD allow reserved characters, as defined by [RFC3986](https://tools.ietf.org/html/rfc3986#section-2.2) `:/?#[]@!$&'()*+,;=` to be included without percent-encoding. The default value is `false`. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`.
+<a name="encodingAllowReserved"></a>allowReserved | `boolean` | When this is true, parameter values are serialized  using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are [not allowed in the query string](https://datatracker.ietf.org/doc/html/rfc3986#autoid-24) (`[`, `]`, `#`), or have a special meaning in `application/x-www-form-urlencoded` (`-`, `&`, `+`); see Appendices [C](#usingRFC6570Implementations) and [E](#percentEncodingAndFormMediaTypes) for details. This property only applies to parameters with an `in` value of `query`. The default value is `false`.
 
 This object MAY be extended with [Specification Extensions](#specificationExtensions).
 
+See also [Appendix C: Using RFC6570 Implementations](#usingRFC6570Implementations) for additional guidance.
+
 ##### Encoding Object Example
 
 `multipart/form-data` allows for binary parts:
@@ -3696,4 +3700,243 @@ Version   | Date       | Notes
 
 ## <a name="usingRFC6570Implementations"></a>Appendix C: Using RFC6570 Implementations
 
+Serialization is defined in terms of RFC6570 URI Templates in two scenarios:
+
+Object | Condition
+------ | ---------
+[Parameter Object](#parameterObject) | When `schema` is present
+[Encoding Object](#encodingObject) | When encoding for `application/x-www-form-urlencoded` and any of `style`, `explode`, or `allowReserved` are used
+
+Implementations of this specification MAY use an implementation of RFC6570 to perform variable expansion, however, some caveats apply.
+
+Note that when using `style: form` RFC6570 expansion to produce an `application/x-www-form-urlencoded` HTTP message body, it is necessary to remove the `?` prefix that is produced to satisfy the URI query string syntax.
+
+Note also that not all RFC6570 implementations support all four levels of operators, all of which are needed to fully support the OpenAPI Specification's usage.
+Using an implementation with a lower level of support will require additional manual construction of URI Templates to work around the limitations.
+
+### Equivalences Between Fields and RFC6570 Operators
+
+Certain field values translate to RFC6570 operators (or lack thereof):
+
+field | value | equivalent
+----- | ----- | ----------
+style | simple | _n/a_
+style | matrix | `;` prefix operator
+style | label | `.` prefix operator
+style | form | `?` prefix operator
+allowReserved | `false` | _n/a_
+allowReserved | `true` | `+` prefix operator
+explode | `false` | _n/a_
+explode | `true` | `*` modifier suffix
+
+Multiple `style: form` parameters are equivalent to a single RFC6570 [variable list](https://www.rfc-editor.org/rfc/rfc6570#section-2.2) using the `?` prefix operator:
+
+```YAML
+parameters:
+- name: foo
+  in: query
+  schema:
+    type: object
+  explode: true
+- name: bar
+  in: query
+  schema:
+    type: string
+```
+
+This example is equivalent to RFC6570's `{?foo*,bar}`, and ***NOT*** `{?foo*}{&bar}`, which is problematic because if `foo` is not defined, the result will be an invalid URI.
+The `&` prefix operator has no equivalent in the Parameter Object.
+
+Note that RFC6570 does not specify behavior for compound values beyond the single level addressed by `explode`.  The results of using objects or arrays where no behavior is clearly specified for them is implementation-defined.
+
+### Non-RFC6570 Field Values and Combinations
+
+Configurations with no direct RFC6570 equivalent SHOULD also be handled according to RFC6570.
+Implementations MAY create a properly delimited URI Template with variables for individual names and values using RFC6570 regular or reserved expansion (based on `allowReserved`).
+
+This includes:
+  * the styles `pipeDelimited`, `spaceDelimited`, and `deepObject`, which have no equivalents at all
+  * the combination of the style `form` with `allowReserved: true`, which is not allowed because only one prefix operator can be used at a time
+  * any parameter name that is not a legal RFC6570 variable name
+
+The Parameter Object's `name` field has a much more permissive syntax than [RFC6570 variable name syntax](https://www.rfc-editor.org/rfc/rfc6570#section-2.3).
+A parameter name that includes characters outside of the allowed RFC6570 variable character set MUST be percent-encoded before it can be used in a URI Template.
+
+### Examples
+
+Let's say we want to use the following data in a form query string, where `formulas` is exploded, and `words` is not:
+
+```YAML
+formulas:
+  a: x+y
+  b: x/y
+  c: x^y
+words:
+- math
+- is
+- fun
+```
+
+#### RFC6570-Equivalent Expansion
+
+This array of parameter objects uses regular `style: form` expansion, fully supported by RFC6570:
+
+```YAML
+parameters:
+- name: formulas
+  in: query
+  schema:
+    type: object
+    additionalProperties:
+      type: string
+  explode: true
+- name: words
+  in: query
+  schema:
+    type: array
+    items:
+      type: string
+```
+
+This translates to the following URI Template:
+
+```urlencoded
+{?formulas*,words}
+```
+
+when expanded with the data given earlier, we get:
+
+```urlencoded
+?a=x%2By&b=x%2Fy&c=x%5Ey&words=math,is,fun
+```
+
+#### Expansion With Non-RFC6570-Supported Options
+
+But now let's say that (for some reason), we really want that `/` in the `b` formula to show up as-is in the query string, and we want our words to be space-separated like in a written phrase.
+To do that, we'll add `allowReserved: true` to `formulas`, and change to `style: spaceDelimited` for `words`:
+
+```YAML
+parameters:
+- name: formulas
+  in: query
+  schema:
+    type: object
+    additionalProperties:
+      type: string
+  explode: true
+  allowReserved: true
+- name: words
+  in: query
+  style: spaceDelimited
+  schema:
+    type: array
+    items:
+      type: string
+```
+
+We can't combine the `?` and `+` RFC6570 prefixes, and there's no way with RFC6570 to replace the `,` separator with a space character.
+So we need to restructure the data to fit a manually constructed URI Template that passes all of the pieces through the right sort of expansion.
+
+Here is one such template, using a made-up convention of `words.0` for the first entry in the words value, `words.1` for the second.
+
+```urlencoded
+?a={+a}&b={+b}&c={+c}&words={words.0} {words.1} {words.2}
+```
+
+RFC6570 [mentions](https://www.rfc-editor.org/rfc/rfc6570.html#section-2.4.2) the use of `.` "to indicate name hierarchy in substructures," but does not define any specific naming convention or behavior for it.
+Since the `.` usage is not automatic, we'll need to construct an appropriate input structure for this new template.
+
+We'll also need to pre-process the values for `formulas` because while `/` and most other reserved characters are allowed in the query string by RFC3986, `[`, `]`, and `#` [are not](https://datatracker.ietf.org/doc/html/rfc3986#appendix-A), and `&`, `=`, and `+` all have [special behavior](https://www.rfc-editor.org/rfc/rfc1866#section-8.2.1) in the `application/x-www-form-urlencoded` format, which is what we are using in the query string.
+
+Setting `allowReserved: true` does _not_ make reserved characters that are not allowed in URIs allowed, it just allows them to be _passed through expansion unchanged._
+Therefore, any tooling still needs to percent-encode those characters because reserved expansion will not do it, but it _will_ leave the percent-encoded triples unchanged.
+See also [Appendix E](#to indicate name hierarchy in substructures) for further guidance on percent-encoding and form media types, including guidance on handling the delimiter characters for `spaceDelimited`, `pipeDelimited`, and `deepObject` in parameter names and values.
+
+So here is our data structure that arranges the names and values to suit the template above, where values for `formulas` have `[]#&=+` pre-percent encoded (although only `+` appears in this example):
+
+```YAML
+a: x%2By
+b: x/y
+c: x^y
+words.0: math
+words.1: is
+words.2: fun
+```
+
+Expanding our manually assembled template with our restructured data yields the following query string:
+
+```urlencoded
+?a=x%2By&b=x/y&c=x%5Ey&words=math%20is%20fun
+```
+The `/` and the pre-percent-encoded `%2B` have been left alone, but the disallowed `^` character (inside a value) and space characters (in the template but outside of the expanded variables) were percent-encoded.
+
+#### Undefined Values and Manual URI Template Construction
+
+Care must be taken when manually constructing templates to handle the values that [RFC6570 considers to be _undefined_](https://datatracker.ietf.org/doc/html/rfc6570#section-2.3) correctly:
+
+```YAML
+formulas: {}
+words:
+- hello
+- world
+```
+
+Using this data with our original RFC6570-friendly URI Template, `{?formulas*,words}`, produces the following:
+
+
+```urlencoded
+?words=hello,world
+```
+
+This means that the manually constructed URI Template and restructured data need to leave out the `formulas` object entirely so that the `words` parameter is the first and only parameter in the query string.
+
+Restructured data:
+
+```YAML
+words.0: hello
+words.1: world
+```
+
+Manually constructed URI Template:
+
+```urlencoded
+?words={words.0} {words.1}
+```
+
+Result:
+
+```urlencoded
+?words=hello%20world
+```
+
+#### Illegal Variable Names as Parameter Names
+In this example, the heart emoji is not legal in URI Template names (or URIs):
+
+```YAML
+parameters:
+- name: ❤️
+  in: query
+  schema:
+    type: string
+```
+
+We can't just pass `❤️: love!` to an RFC6570 implementation.
+Instead, we have to pre-percent-encode the name (which is a six-octet UTF-8 sequence) in both the data and the URI Template:
+
+```YAML
+"%E2%9D%A4%EF%B8%8F": love!
+```
+
+```urlencoded
+{?%E2%9D%A4%EF%B8%8F}
+```
+
+This will expand to the result:
+
+```urlencoded
+?%E2%9D%A4%EF%B8%8F=love%21
+```
+
 ## <a name="serializingHeadersAndCookies"></a>Appendix D: Serializing Headers and Cookies
+
+## <a name="percentEncodingAndFormMediaTypes"></a>Appendix E: Percent-Encoding and Form Media Types