ipip(0445): add skip-leaves

Author: Jorropo

src/http-gateways/trustless-gateway.md

@@ -214,7 +214,7 @@ The Body hash MUST match the Multihash from the requested CID.
 
 A CAR stream for the requested
 [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car)
-content type (with optional `order` and `dups` params), path and optional
+content type (with optional `order`, `dups` and `skip-leaves` params), path and optional
 `dag-scope` and `entity-bytes` URL parameters.
 
 ## CAR version
@@ -301,6 +301,26 @@ of their presence in the DAG or the value assigned to the "dups" parameter, as
 the raw data is already present in the parent block that links to the identity
 CID.
 
+## CAR `skip-leaves` (content type parameter)
+
+The `skip-leaves` parameter specifies whether blocks with the multicodec `raw`
+`0x55` must be sent.
+
+It accepts two values:
+- `y`: Blocks with `raw` multicodec MUST NOT be sent.
+- `n`, or unspecified: Blocks with `raw` multicodec MUST be sent.
+
+A gateway MUST NOT assume this field is `y` if unspecified.
+When not specified it always MUST be understood as `n`.
+
+:::note Notes for implementers
+
+A request which is rooted at a `raw` block and has `skip-leaves=y` does not
+make sense and SHOULD NOT be sent by clients, it is fair for servers to
+error in this situation.
+
+:::
+
 ## CAR format parameters and determinism
 
 The default header and block order in a CAR format is not specified by IPLD specifications.

src/ipips/ipip-0445.md

@@ -0,0 +1,105 @@
+---
+title: "IPIP-0445: trustless gateway skip-leaves option"
+date: 2023-10-09
+ipip: open
+editors:
+  - name: Hugo VALTIER
+    github: Jorropo
+    url: https://jorropo.net/
+    affiliation:
+        name: Protocol Labs
+        url: https://protocol.ai/
+relatedIssues:
+  - https://github.com/ipfs/specs/issues/444
+order: 445
+tags: ['ipips']
+---
+
+## Summary
+
+Introduce `skip-leaves` flag for the :cite[trustless-gateway].
+
+## Motivation
+
+Allow clients to read a stream which only contain proofs in a bottom heavy
+graph using `raw` codec for it's leaves.
+
+Usefull with unixfs for features like webseeds [#444](https://github.com/ipfs/specs/issues/444).
+
+## Detailed design
+
+The `skip-leaves` CAR Content-Type parameter on :cite[trustless-gateway]
+allows clients to download an entity except blocks with the multicodec
+`raw` (`0x55`).
+
+- When set to `y`, the parameter instructs the gateway not to transmit
+  blocks tagged with the `raw` multicodec.
+- If set to `n`, or left unspecified, the gateway MUST transmit `raw`
+  multicodec blocks.
+
+Importantly, unless explicitly specified as `y`, the default operational
+mode of the gateway MUST assume the value of `skip-leaves` to be `n`.
+
+## Design rationale
+
+### User Benefit
+
+Implementing the `skip-leaves` parameter offers several benefits to users:
+
+1. **Verification Flexibility:** Clients can verify out-of-band (OOB) received
+   files in their deserialized form without necessitating the transmission of
+   raw blocks from the gateway.
+2. **Incremental Download:** Clients can incrementally download files in
+   deserialized forms from non-IPFS servers. Allowing applications to share
+   distribution for IPFS and non IPFS clients.
+3. **Efficient Block Discovery:** With the `skip-leaves` option enabled,
+   clients can quickly discover numerous candidate blocks without being
+   bottlenecked by the gateway's transmission of raw blocks.
+
+### Compatibility
+
+Setting the default value of the `skip-leaves` parameter to `n` ensures
+backward compatibility with existing clients and systems that are unaware
+of this new flag.
+
+### Prevention of Amplification Attacks and Efficient Server Operation
+
+By utilizing the `raw` (`0x55`) codec servers can trivially determine whether
+to fetch or skip a block without having to learn any new information.
+Although more limited and not able to handle unixfs file using dag-pb for their
+leaves, it allows both the client and server to trivially verify a block
+must not be fetched. Preventing issues of Amplification where a server could
+need to fetch multiple orders more data than the client when executing the
+request.
+
+### Why not `dag-scope=skip-leaves` ?
+
+The `dag-scope` parameter determines the overall range of blocks to retrieve,
+while `skip-leaves` selectively filters specific blocks within that range.
+Combining them under one parameter would restrict their combined utility.
+
+For example:
+- A client is streaming a video from a webseed and the user seeked through the
+  video, then the client would send `dag-scope=entity&entity-bytes=42:1337`
+  with `skip-leaves=y` to download the proofs for the required section of the
+  video.
+- A client is verifying an OOB transfered directory, then `dag-scope=all` with
+  `skip-leaves=y` makes sense.
+
+### Alternatives
+
+An alternative approach would be to request blocks individually.
+However it adds extra round trips and more per HTTP request overhead
+and thus is undesireable.
+
+## Security
+
+None.
+
+## Test fixtures
+
+TODO
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).