Path queries (feature 66/67)

[brh28]

Path Queries

Introduction

To route a payment on the Lightning Network, a sender must find a path to the destination using channels which contain sufficient liquidity and meet certain routing rules (e.g fees). The current gossip scheme is insufficient to reliably determine a feasible path and inflexible for routing nodes. The purpose of path queries is to reduce the informational requirements during pathfinding and to allow routers to respond with dynamic policy. By selectively sharing routing information between peers, payment reliability can be scaled to a growing network while preserving channel balance privacy and payment anonymity.

The problem: Graph Dependence

While finding a feasible path, source-based routing requires information about the network graph. For Lightning, this information typically comes from two sources: gossip messages and the responses of previous payment attempts. Both sources have severe limitations, which ultimately favor larger routing nodes and contribute to routing centralization.

Limitations of gossip

The gossip protocol is characterized by it's ability to quickly and reliably deliver a finite amount of information across a large distributed system via propagation. However, because these messages are delivered to every node (potentially multiple times), node performance suffers with a growing quantity of data; the more data that's shared, the more network and computational resources are required by each node to process those messages. Therefore, the size and frequency of gossip messages must be constrained. Gossip is well-suited for the Bitcoin network because nodes seek consistency among each other, which is done by limiting transaction throughput. Data on the Lightning Network, however, is more dynamic and routing a payment does not require consistency among all nodes, which means message limits and standardness rules are unnecessarily restrictive. For example, routing policy (cltv delta, fees, etc) has highly dynamic inputs, including available liquidity, HTLC slots, onchain fees, and even external factors. Existing limits to channel_update messages prohibit nodes from accurately reflecting their desired policy.

Finding Liquidity

For a payment to succeed, a route requires sufficient liquidity in every channel of the path. From a data perspective, liquidity is state that is managed between two nodes. During pathfinding, the unpredictability of a channel's liquidity is referred to as liquidity uncertainty. Without any prior knowledge (i.e high uncertainty), the probability a path is feasible declines with the size of the payment and the number of channels used. Today, feasible paths are found by a process of trial-and-error, whereby liquidity ranges are temporarily narrowed using the results of previous payment attempts, but this approach has a host of issues:

1. When liquidity uncertainty is high, pathfinding calculations increase payment success probabilities by favoring shorter paths and higher capacity channels. Not only does this affect the payment sender who's more likely to pay extra in fees for reliable liquidity, it's also a centralizing force on the network.

2. Lower payment success probabilities implies a larger set of potential routes. When the final route is uncertain, routing fees (and other payment details) are more difficult to predict.

3. Trial-and-error is a slow discovery process because:

   a. HTLCs require multiple message exchanges between peers, with each HTLC addition and removal requiring commitment/revocation cycles.

   b. Payments must be attempted serially to avoid the delivery of multiple successful payments.

   To improve performance for real payments, nodes may choose to 'probe' the channels of routing nodes using fake payments. However, liquidity is often highly dynamic and is always regressing to a state of uncertainty. To be effective, nodes must actively monitor the network.

4. Failed payments (both real and fake) are a burden to routing nodes in the failing sub-path in the form of locked liquidity, HTLC slots, and wasted system resources.

Furthermore, each of these problems represent a scaling constraint, as more nodes need to search for liquidity amongst a larger set of channels.

Proposal

The goal of path queries is to reduce a node's dependence on the graph during pathfinding by leveraging the routing information of other nodes. Specifically, this feature includes the following optional messages which allows nodes to cooperatively construct a path:

1. query_path

• source_node_id

• destination_node_id

• amount

2. reply_path

• path

3. reject_query_path

• reason

Upon receiving a query_path message, a node can choose how it wants to respond, including rejecting or ignoring it. The reply_path message helps the requester - either a source or router - deliver a potential payment because it leverages routing information at the queried node. This resolves the liquidity uncertainty problem at the queried hop because a node knows its own channel balances and can respond accordingly. A routing node can respond with any routing policy (e.g fees, expiry, etc) it desires, unconstrained by limits to gossip. Compared to payments, queries are lightweight and can be made concurrently.

A full summary can be found in proposals/path-queries.md

[t-bast]

I don't really understand what path queries would bring since we already have trampoline. I believe you may also be over-estimating trampoline's complexity, which may be why you're looking for an alternative solution? FWIW the reason trampoline has been slow to be adopted is not because of its perceived complexity, but rather because such routing improvements are low on everyone's priority list (because in practice, payments work well) and that most implementations aren't focused on wallets. Path queries would thus have the same issue (even more so since trampoline is almost done on the LDK side and in progress in cln).

On top of that, I think that the main issue with path queries is that DoS policies are going to be incredibly hard to implement. The spec may look simple, but that doesn't mean that a safe implementation will be...

[brh28]

Replacing trampoline isn't the primary goal, but an emergent benefit. The proposal describes the full benefits in greater depth, but I'll add that routing nodes also benefit because:

1. They are less dependent on payment volume/probing (good for small nodes)
2. They are less likely to be probed (good for big nodes)
3. They can potentially use this as a mechanism to mitigate channel jamming attacks.
4. They do not need the full graph if they have a supporting peer, though a sub-graph or some representation of the full graph would likely be useful.
5. They can update their routing policy for each query. More dynamic policy => more liquidity control

Regarding DoS policies, I'm hoping you could elaborate on your concerns. Hypothetically, if I defined a policy with a rate limit of 5 responses to a query source per minute and respond only to peers, how would you attack me?

[yyforyongyu]

DoS prevention can be hard as it involves tradeoffs - if you rate limit too heavily, sending payments will be slow; otherwise you are vulnerable to being DoSed. Tho if you have a channel with a peer, you are less likely to abuse them. I think the main concern comes to the actual implementation - suppose I have an MMP, I now need to do a full query/response cycle to get a path for a particular HTLC, which can slow down the process significantly. Then there's the question of how to split the full amount into HTLCs. On the other hand, what is the incentive for a node to offer this service? And what if they decide to give us false info to mess with us? Actually if there's a failure in sending an HTLC, how do we know it's because of the false info or some other nodes down the path went offline? The addition may look simple, but the implementation is not.

[brh28]

DoS prevention can be hard as it involves tradeoffs

Lightning is well-equipped to handle DoS attacks:

1. It has built-in authentication, making it easy to detect the source of malicious traffic.
2. It supports micro-payments, making it easy to pay for routing services.

I would expect routing nodes to develop dynamic rate limits; the more successful payments you send through my node, the more queries I respond to.

I now need to do a full query/response cycle

Yes, you need to do one round-trip to eliminate uncertainty at that hop. How is this worse than a failing payment that requires an HTLC with multiple rounds of communication and only reduces uncertainty to the attempted amount?

how to split the full amount into HTLCs

This is already a decision payment senders have to make for the full route. With path queries, routing nodes can tell the sender how to more effectively split the payment.

what is the incentive for a node to offer this service?

Routing fees.

what if they decide to give us false info to mess with us?

They could, but lying is bad for business. They would be missing potential fees and damaging their reputation when that path fails. Can you think of a case when that might not be true?

if there's a failure in sending an HTLC, how do we know it's because of the false info or some other nodes down the path went offline?

Failed payments return the same errors.

[brh28]

@t-bast I updated the proposal to hopefully clarify the motivations and rewrote the section on trampoline, which I've now titled "Comparisons to Trampoline," since path queries are functionally different. I'd love to get your thoughts here to make sure it's a fair comparison.

[EDIT]: I iterated on that section again. Hope it doesn't cause any confusion :)