HTLCs claimed by remote remain in pending claim requests

[whfuyn]

This logs recur frequently. The node persistently attempts to claim HTLCs already spent by counterparty, resulting in repeated failed fee bumps.

2025-06-12T10:42:36.056259Z DEBUG lightning_transaction_sync::esplora:248: Finished transaction sync at tip 2b363f57ae69d3fcaf3daa49087b23f665a2ce406bd3d3d93bea355575e8ee75 in 118ms: 0 confirmed, 0 unconfirmed.
2025-06-12T10:42:36.056288Z  INFO ldk_node::chain:592: Sync of Lightning wallet finished in 118ms.
2025-06-12T10:42:39.006940Z  INFO lightning::chain::onchaintx:513: Triggering rebroadcast/fee-bump for request with inputs [OutPoint { txid: a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a, vout: 3 }, OutPoint { txid: a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a, vout: 8 }, OutPoint { txid: a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a, vout: 6 }]
2025-06-12T10:42:39.006983Z  INFO lightning::chain::onchaintx:513: Triggering rebroadcast/fee-bump for request with inputs [OutPoint { txid: a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a, vout: 5 }]
2025-06-12T10:42:39.006987Z  INFO lightning::chain::onchaintx:513: Triggering rebroadcast/fee-bump for request with inputs [OutPoint { txid: a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a, vout: 4 }, OutPoint { txid: a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a, vout: 7 }]
2025-06-12T10:42:39.006997Z DEBUG lightning::chain::package:1341: Initiating fee rate bump from 1570 s/kWU (2339 s) to 253 s/kWU (376 s) using HighestOfPreviousOrNew strategy
2025-06-12T10:42:39.044761Z DEBUG lightning::chain::package:1062: Adding claiming input for outpoint a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a:3
2025-06-12T10:42:39.045041Z DEBUG lightning::chain::package:1062: Adding claiming input for outpoint a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a:8
2025-06-12T10:42:39.045336Z DEBUG lightning::chain::package:1062: Adding claiming input for outpoint a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a:6
2025-06-12T10:42:39.045591Z DEBUG lightning::chain::onchaintx:526: Generated claim transaction HTLC claim tx (0 preimage, 3 timeout, 0 revoked) with txid 264ea0c5d6300b5dbcf276818c75596958cf14a51b253fb075b919152d38b7c0 with feerate 1570
2025-06-12T10:42:39.045609Z  INFO lightning::chain::onchaintx:530: Rebroadcasting onchain HTLC claim tx (0 preimage, 3 timeout, 0 revoked) with txid 264ea0c5d6300b5dbcf276818c75596958cf14a51b253fb075b919152d38b7c0
2025-06-12T10:42:39.045623Z DEBUG lightning::chain::package:1341: Initiating fee rate bump from 754 s/kWU (544 s) to 253 s/kWU (182 s) using HighestOfPreviousOrNew strategy
2025-06-12T10:42:39.082771Z DEBUG lightning::chain::package:1062: Adding claiming input for outpoint a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a:5
2025-06-12T10:42:39.083042Z DEBUG lightning::chain::onchaintx:526: Generated claim transaction HTLC claim tx (0 preimage, 1 timeout, 0 revoked) with txid 51181b7b7efd091a5be5294e6d6448f699f02ff8ff14b2fcb1aa27039c899438 with feerate 754
2025-06-12T10:42:39.083060Z  INFO lightning::chain::onchaintx:530: Rebroadcasting onchain HTLC claim tx (0 preimage, 1 timeout, 0 revoked) with txid 51181b7b7efd091a5be5294e6d6448f699f02ff8ff14b2fcb1aa27039c899438
2025-06-12T10:42:39.083066Z DEBUG lightning::chain::package:1341: Initiating fee rate bump from 1258 s/kWU (1391 s) to 253 s/kWU (279 s) using HighestOfPreviousOrNew strategy
2025-06-12T10:42:39.128809Z DEBUG lightning::chain::package:1062: Adding claiming input for outpoint a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a:4
2025-06-12T10:42:39.129652Z DEBUG lightning::chain::package:1062: Adding claiming input for outpoint a497f47f05a42cc0bd83a7b80e2ef6cc05e47c3a1a7592adf97adcbbd17c344a:7
2025-06-12T10:42:39.130057Z DEBUG lightning::chain::onchaintx:526: Generated claim transaction HTLC claim tx (0 preimage, 2 timeout, 0 revoked) with txid a58e781b8a852fef47ae9ebaab21375455f9b75d4dd630daf7392c7d7031b5a9 with feerate 1258
2025-06-12T10:42:39.130081Z  INFO lightning::chain::onchaintx:530: Rebroadcasting onchain HTLC claim tx (0 preimage, 2 timeout, 0 revoked) with txid a58e781b8a852fef47ae9ebaab21375455f9b75d4dd630daf7392c7d7031b5a9
2025-06-12T10:42:39.131245Z DEBUG ldk_node::chain:1036: Failed to broadcast due to HTTP connection error: RPC error
tx: 020000000001034a347cd1bbdc7af9ad92751a3a7ce405ccf62e0eb8a783bdc02ca4057ff497a40300000000010000004a347cd1bbdc7af9ad92751a3a7ce405ccf62e0eb8a783bdc02ca4057ff497a40800000000010000004a347cd1bbdc7af9ad92751a3a7ce405ccf62e0eb8a783bdc02ca4057ff497a4060000000001000000029502000000000000160014c677f924d93621a79916596d1821eadb3d7f67870000000000000000226a20232dde9874fa0f7b80ff46f16ba93e1109f1ab49e8fb219f82fac09dfe86ad3403473044022072af87b9f7233425d3ba33057b2fae07b417fc1af1866847497dd238e140ee8802201189b6586d51251480b1c105903d83c34efdcc2324164e01745cc5fefc17a16f01008d76a9144c674883e5d084c88dabbdef1bf212e015239b468763ac672102a151fe4212fdff1f68c5d3b8f1a5d4d7b73aa5803676f33766081d3fc687bc4a7c8201208763a9144613933d00e9c002b1d63ea05908eb9ed21f198188527c2103edb96e973a500b364169064e8f0dc33e797222cb9133551c7717707a576fb1da52ae6775027e25b175ac6851b275680347304402203cd63df8fd56ce14708e0ca16840433fdde1412acac9c70d6873c41ad9909d0b022074d879b2285f4f54cc2f77d64888adc59024b3c9c28182f299377165095df13e01008d76a9144c674883e5d084c88dabbdef1bf212e015239b468763ac672102a151fe4212fdff1f68c5d3b8f1a5d4d7b73aa5803676f33766081d3fc687bc4a7c8201208763a914ae7b315df8c3443980affd5fd9907629dd84b32088527c2103edb96e973a500b364169064e8f0dc33e797222cb9133551c7717707a576fb1da52ae6775027e25b175ac6851b275680347304402201278483492cd936f68b4c3df3d8c07c87c6ef23a6a02955b51552f36a873db17022034a902d8f2ca07504819910881e908555576846169d3f6e3d7fd84c6df9b1a0801008d76a9144c674883e5d084c88dabbdef1bf212e015239b468763ac672102a151fe4212fdff1f68c5d3b8f1a5d4d7b73aa5803676f33766081d3fc687bc4a7c8201208763a914fe3ed73f755ba24f8b62ccb5cff0d8c6f1b5400788527c2103edb96e973a500b364169064e8f0dc33e797222cb9133551c7717707a576fb1da52ae6775027e25b175ac6851b27568f8250000
2025-06-12T10:42:39.131877Z DEBUG ldk_node::chain:1036: Failed to broadcast due to HTTP connection error: RPC error
tx: 020000000001014a347cd1bbdc7af9ad92751a3a7ce405ccf62e0eb8a783bdc02ca4057ff497a405000000000100000002c801000000000000160014c677f924d93621a79916596d1821eadb3d7f67870000000000000000226a20867360ae650cb132965b8ba1a3ea02323ab8d359632036e72e98ac9fe33f06840347304402206d36df39dfaadb21652ff9fa0690ee18cc2adb7ed050073bea835d858ab867a5022052d1b7945f27e07c65ab17ccb5e926965ce4571bd4eac9226c3241fa0492c43201008d76a9144c674883e5d084c88dabbdef1bf212e015239b468763ac672102a151fe4212fdff1f68c5d3b8f1a5d4d7b73aa5803676f33766081d3fc687bc4a7c8201208763a91433531d3e62ee2467835b6836fd027bd66d2cdd1088527c2103edb96e973a500b364169064e8f0dc33e797222cb9133551c7717707a576fb1da52ae677502a625b175ac6851b27568f8250000
2025-06-12T10:42:39.132663Z DEBUG ldk_node::chain:1036: Failed to broadcast due to HTTP connection error: RPC error
tx: 020000000001024a347cd1bbdc7af9ad92751a3a7ce405ccf62e0eb8a783bdc02ca4057ff497a40400000000010000004a347cd1bbdc7af9ad92751a3a7ce405ccf62e0eb8a783bdc02ca4057ff497a4070000000001000000026102000000000000160014c677f924d93621a79916596d1821eadb3d7f67870000000000000000226a20785ae3c2af987f6918de7e97c31a134034fb1ba4f452365b513ba93f24d657980347304402207f15669b612c314338cce5d9f69e779a17bdd7088cdd1fa83b77ff9ce14886c102200c9357b8761c1335800a7864ed6b6a0c0e682f3df4e2de4ee78900ae188d262101008d76a9144c674883e5d084c88dabbdef1bf212e015239b468763ac672102a151fe4212fdff1f68c5d3b8f1a5d4d7b73aa5803676f33766081d3fc687bc4a7c8201208763a914f3937f738169c949098699bc0b06ad1efa49bf8b88527c2103edb96e973a500b364169064e8f0dc33e797222cb9133551c7717707a576fb1da52ae6775025625b175ac6851b275680347304402200bd146777a2919cd506df470bfeedc662bc92c955587a99f834395a7ef4ff1ab02200ec384b0693c8916b3065ada55fa53a3fecd2b0227115194125c87921f52eb2501008d76a9144c674883e5d084c88dabbdef1bf212e015239b468763ac672102a151fe4212fdff1f68c5d3b8f1a5d4d7b73aa5803676f33766081d3fc687bc4a7c8201208763a91421997e3bd84561a3a9a3942c39891c6667afbb1a88527c2103edb96e973a500b364169064e8f0dc33e797222cb9133551c7717707a576fb1da52ae6775025625b175ac6851b27568f8250000

This issue is timing-dependent and subtle. The root cause is in the lockedtimed_packages.

Scenario Reproduction:
Assume two HTLCs (1 and 2) on counterparty's commitment transaction, which they broadcast:

• HTLC 1 lock time: Block 100
• HTLC 2 lock time: Block 102

Sequence of events:

1. Block 100 confirmed:
   • HTLC 1 added to our pending claim requests
2. Block 101 confirmed:
   • Counterparty's transaction spending both HTLCs 1 and 2 confirms
   • Spending notifications received → HTLC 1 removed from pending claim requests
   • Chain source stop monitoring both HTLCs
     • because both of watched outputs are in the same transaction
3. Block 102 confirmed:
   • HTLC 2 added to pending claim requests (lock time met)
   • Problem: No spending notification for HTLC 2 occurs until node restart (when watched outputs reload)

Consequence:
The node continues attempting to claim the already-spent HTLC 2, triggering unnecessary fee-bump attempts.

If spent HTLCs are included in a claim request alongside unspent HTLCs, the entire claim fails, preventing other HTLCs from being claimed.