[feature]: more advanced channel peer IP address management

[ZZiigguurraatt]

When we have a channel open with a peer, we need to have a direct connection to that peer. Some peers advertise their IP address publicly via a NodeAnnouncement message that comes through the peer to peer gossip. Some nodes never provide a NodeAnnouncement message with their IP address and the channel peer receives that information out of band or when the channel peer connects to them (from the source IP address). Also, at any time a node can stop advertising their IP address. Therefor, LND must also keep track of the IP address it connected to a peer with when the channel was opened so that it can later reconnect to the peer if they remove their IP address from their NodeAnnouncement message or never provided a NodeAnnouncement message.

Here is some more details provided by claude code:

❯ where in the DB does lnd store peer IP addresses for channels                                                                                                                                                                                                                                                               

  Thought for 4s (ctrl+o to expand)

● I'll search the lnd codebase to find where peer IP addresses are stored in the database.

● Explore(Find peer IP address DB storage)
  ⎿  Done (38 tool uses · 43.4k tokens · 2m 10s)
  (ctrl+o to expand)

● lnd stores peer IP addresses in two distinct databases, and for channel peers specifically the primary store is the LinkNode record.

  1. Primary store for channel peers — LinkNode (channeldb)

  When you have (or had) a direct channel with a peer, lnd persists their network addresses in a LinkNode record in the channel state DB (channel.db).

  - Struct: channeldb/nodes.go:25
  type LinkNode struct {
      Network     wire.BitcoinNet
      IdentityPub *btcec.PublicKey
      LastSeen    time.Time
      Addresses   []net.Addr   // ← the peer's IP/TCP/onion addresses
      db          *LinkNodeDB
  }
  - Bucket: "nib" (nodeInfoBucket) — channeldb/nodes.go:22
  - Key: the peer's 33-byte compressed pubkey → Value: serialized LinkNode (network, pubkey, last-seen, then the address list)
  - Key functions: putLinkNode, FetchLinkNode, FetchAllLinkNodes, and AddAddress/Sync to append/persist new addresses.

  2. Secondary store — graph node announcements (graph/db)

  Every node in the routing graph (including your channel counterparties) also has addresses from its gossip node_announcement, stored separately:

  - Struct: Node.Addresses — graph/db/models/node.go:19
  - Bucket: "graph-node" — graph/db/kv_store.go:44
  - Key: 33-byte pubkey → serialized node announcement data including addresses.

  Address encoding

  Both stores share the same serializer (SerializeAddr/DeserializeAddr in graph/db/addr.go), with a 1-byte type tag: IPv4 (0), IPv6 (1), Tor v2/v3 onion (2/3), DNS (5), opaque (4).

  How it ties together on reconnect

  On startup, server.go (~line 3658) calls FetchAllLinkNodes() to get every channel peer's stored addresses, and both LinkNodeDB and GraphDB satisfy a common AddrSource interface (AddrsForNode, channeldb/addr_source.go) so the connection logic can union addresses from both sources before dialing the peer.

  Bottom line: For channel peers, the authoritative location is the "nib" bucket in channel.db (the LinkNode.Addresses field). The graph's "graph-node" bucket is a supplementary source populated from gossip.

✻ Cooked for 2m 35s

Also, there is currently no way to remove a stale IP address that was associated with a channel peer. Here is some more information about this provided by claude code:

❯ how are IP addresses removed frrom channeldb?                                                                                                                                                                                                                                                                               

● I'll look into how addresses are removed from the LinkNode storage in channeldb.

  Read 1 file (ctrl+o to expand)

● Now let me check who calls DeleteLinkNode to understand the lifecycle, and confirm there's no per-address removal.

  Searched for 1 pattern, read 1 file (ctrl+o to expand)

● Let me see the rest of PruneLinkNodes and find who calls it.

  Read 1 file (ctrl+o to expand)

● Here's how IP addresses get removed from channeldb.

  Key point: there is no per-address removal

  The LinkNode API only ever adds addresses (AddAddress at channeldb/nodes.go:91) or overwrites the whole list via Sync (putLinkNode does a full bucket.Put). There is no function to delete an individual IP from a peer's address list. Addresses disappear only when the entire LinkNode record is deleted from the "nib"
  bucket.

  The actual deletion mechanism

  deleteLinkNode — channeldb/nodes.go:236 — does a nodeMetaBucket.Delete(pubKey), wiping the whole record (network, last-seen, and all addresses) keyed by the peer's compressed pubkey.

  It's reached three ways:

  1. On channel close, when it was the last channel with that peer — channeldb/db.go:1506. After writing the close summary, lnd checks fetchOpenChannels; if zero remain, it garbage-collects the link node in the same transaction (to avoid a race where a new channel opens between check and delete). This is the normal
  path: once you have no open channels with a peer, lnd stops persisting their addresses so it won't keep reconnecting.
  2. pruneLinkNode — channeldb/db.go:1523 — the same logic factored out, with a defensive double-check for open channels inside the write transaction.
  3. PruneLinkNodes (bulk) — channeldb/db.go:1556 — iterates all link nodes and prunes every one that has no open channels.
  4. DeleteLinkNode — channeldb/nodes.go:230 — the public single-peer wrapper.

  Summary of the lifecycle

  - Add: AddAddress / Sync → putLinkNode (Put)
  - Update/replace: Sync overwrites the full serialized record
  - Remove: only whole-record deletion via deleteLinkNode (Delete), triggered by closing the last channel with a peer, or by explicit/bulk pruning of peers with zero open channels

  So an IP address is never individually evicted — it's discarded as a side effect of the peer's LinkNode being garbage-collected once no open channels remain. (Note also RepairLinkNodes at db.go:1601, which re-creates link nodes that were erroneously deleted by the old pruning race, but that re-adds records using
  channel/peer data rather than removing addresses.)

✻ Crunched for 26s

※ recap: You're exploring how lnd stores and manages peer IP addresses in channeldb. I've explained that addresses live in LinkNode records (the "nib" bucket) and are removed only when the whole record is deleted after a peer's last channel closes. Awaiting your next question. (disable recaps in /config)

When connecting, there is no priority to what known IP addresses are used. All addresses are tried at the same time. Here is some more information about this provided by claude code:

❯ please explain the priority of address selection between the different sources                                                                                                                                                                                                                                              

  Thought for 4s (ctrl+o to expand)

● I'll investigate how lnd combines and prioritizes addresses from the different sources when connecting to peers.

  Read 1 file (ctrl+o to expand)

● Now let me see how this is constructed and how the addresses are used when connecting. Let me look at the server.go usage.

  Searched for 1 pattern, read 1 file (ctrl+o to expand)

● So the order in NewMultiAddrSource is: dbs.ChanStateDB (LinkNode / channeldb) first, then v1Graph (graph node announcements). But the dedup map doesn't preserve priority by source order in a meaningful way for selection — let me check how the resulting address list is actually consumed when dialing, since that's
  where "priority" really matters.

  Read 1 file (ctrl+o to expand)

● This is the heart of the priority logic. Let me read a bit further to see how the addresses then get handed to the connection manager (which dictates dial order).

  Read 1 file (ctrl+o to expand)

● Now let me check how the connection manager picks which address to dial when there are multiple, since the final dial order matters too.

  Searched for 1 pattern, read 1 file (ctrl+o to expand)

● I now have the complete picture. Here's how address-source priority actually works in lnd.

  The short answer: there is no real "priority" — sources are unioned and de-duplicated, not ranked

  lnd does not rank one source above another and pick the "best" address. It gathers addresses from all sources, de-dupes them by string, and then tries to connect to all of them in parallel (staggered). Whichever one succeeds first wins. So the only thing "source order" affects is which duplicate object survives
  de-duplication — and since duplicates are identical addresses, that's immaterial to connectivity.

  There are two layers where addresses get combined, and they behave slightly differently:

  Layer 1 — multiAddrSource (the AddrsForNode query path)

  channeldb/addr_source.go:43, constructed at server.go:781:
  addrSource := channeldb.NewMultiAddrSource(dbs.ChanStateDB, v1Graph)
  Order is LinkNode (channeldb) first, then graph. But AddrsForNode collects into a map[string]net.Addr keyed by addr.String() (addr_source.go:52,70). Because the graph is queried second, on a collision the graph's net.Addr object overwrites the LinkNode one. They're equal addresses, so this has no functional effect
  — and the final map iteration order is random anyway. known is just an OR across sources.
  
  This path is used for on-demand reconnects (e.g. server.go:1750, 2838).

  Layer 2 — establishPersistentConnections (startup reconnect)

  This path (server.go:3649) is hand-rolled and is where you can actually see the intended ordering, though it still ends in a union:

  1. LinkNode addresses are seeded first into nodeAddrsMap (server.go:3663-3670), filtered through withoutV2Onion.
  2. For each channel peer in the graph, it builds an addrSet starting from the graph/NodeAnnouncement addresses (3695), then adds any LinkNode addresses not already present (3713-3735) — so within a peer, graph and link-node addrs are merged, deduped by string.
  3. Crucially, the combined graph result then overwrites the link-node-only entry: nodeAddrsMap[pubStr] = nodeAddr (3774-3776). So for any peer that exists in the graph, the final set = graph ∪ linkNode. For a peer not in the graph, it keeps the LinkNode-only set from step 1.

  Filtering rules applied during the merge (these matter more than "priority")

  - v2 onion addresses are always dropped (withoutV2Onion, and isV2OnionAddr checks at 3696, 3718).
  - Tor (v3 onion) addresses are only included if cfg.Tor.Active (3706-3709, 3728-3732).
  - Only *net.TCPAddr and *tor.OnionAddr types survive; anything else is silently skipped.

  Final dial behavior — all addresses tried concurrently, order non-deterministic

  The merged list goes into s.persistentPeerAddrs (server.go:3809), then connectToPersistentPeer (server.go:4984) loads them into another map[string] (addrMap, 4993) and fires a connmgr.Connect for every remaining address (4045-4063), spaced by multiAddrConnectionStagger. Because it iterates a Go map, the dial order
  is effectively random, not source-ordered. The connection manager races them; the first successful brontide handshake establishes the peer and the rest become redundant.

  Summary

  ┌─────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────┐
  │         Aspect          │                                       Behavior                                        │
  ├─────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
  │ Combination model       │ Union + de-dup by addr.String(), not ranked selection                                 │
  ├─────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
  │ Nominal source order    │ LinkNode (channeldb) seeded first, graph second/overwriting                           │
  ├─────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
  │ Effect of that order    │ None functionally — collisions are identical addresses; final iteration is map-random │
  ├─────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
  │ Real "priority" filters │ Drop v2 onion always; include v3 onion only if Tor active; keep only TCP/Onion types  │
  ├─────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
  │ Dial strategy           │ Connect to all surviving addresses concurrently (staggered); first success wins       │
  └─────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────┘

  So if you were hoping lnd prefers, say, the freshest gossip address over a stale LinkNode address, it does not — both are dialed simultaneously and the network decides.

✻ Worked for 48s

※ recap: You're exploring how lnd handles peer IP addresses in channeldb: where they're stored, removed, and prioritized across sources. All three questions are answered; no code changes pending. Next, tell me what you'd like to dig into. (disable recaps in /config)

Currently we see what IP address a peer advertises via NodeAnnouncement using

• lncli describegraph
• lncli getnodeinfo

however, there is no RPC that exposes the non advertised IP addresses that LND persists for channel peers.

What I'd like to see:

• An RPC to list persisted IP addresses associated with a channel peer that are not part of the NodeAnnouncement, as well as the last known connection time/date with each address.
• An RPC to remove a IP address associated with a channel peer in case we have an address that still works, but we don't want it to be used anymore because we know it is a lower quality route than another address, we know the address will be depreciated soon, etc..
• An RPC to manually add a persisted IP address associated with a channel peer. Maybe fixing [feature]: make lncli connect --perm survive restarts #10870 is good enough?
• Rather than try all, LND prioritize connections by ranking IP addresses based on their last known successful connection and try those first. This would automatically de-prioritize stale addresses without requiring users to manually remove them.

[github-actions]

Found 1 possible duplicate issue:

1. Can't update IP for persistent channel connection #2340

If this issue is a duplicate, please close it and 👍 the existing issue instead.

🤖 Generated with Claude Code

[ZZiigguurraatt]

We likely don't want to work on this issue until after we migrate the channel DB from KV to native SQL.

[murraystewart96]

@ZZiigguurraatt. I've spent some time trying to understand this issue. From what I gather the main issue you are concerned about is that if we have a channel open with another node we might lose knowledge of its IP and not be able to re-establish the channel.

When a channel is initially established both nodes persist the IP of their partner. The issue then becomes what if the partner to our node changes its IP and then stops announcing its IP and the connection drops.

At this point we no longer know the IP of the partner. But the partner knows our IP, so if their node came back online it would connect to our node. At this point we are relying on them to connect to us. This seems reasonable to me because if they were not reconnecting it would likely be because they are offline and we wouldn't be able to connect to them anyway.

Without being able to dynamically discover the new IP for the private node we have to rely on the node re-establishing the connection with us. One thing we could add that we don't have now is that when the partner re-connects we don't store its new IP address. We only store the IP address when the initial channel connection is made. We could make a small change to store the LinkNode address when channels are re-established. Usually public nodes announce their IP when they reconnect so this isn't an issue. But this is a small edge case for private nodes. With knowledge of their new IP, if the connection drops again, both us and the other node can initiate the connection.

However my feeling is that this doesn't give us much benefit given that we need them to be online to make a connection and if they are online they will make the connection to us. Although for correctness maybe we should add this feature.

Please let me know if I have any errors in my thinking.

[ZZiigguurraatt]

One thing we could add that we don't have now is that when the partner re-connects we don't store its new IP address.

This is a good idea and would help too. I'm wondering if that should be a separate issue though, or combined with #10870 ?

[ZZiigguurraatt]

At this point we no longer know the IP of the partner.

If they have a private channel with us, we may already have some out of band communication with the peer. In that case, that's why I was proposing #10870 so that we can persist the new IP address that was received through some private out of band communication.

[ZZiigguurraatt]

At this point we no longer know the IP of the partner. But the partner knows our IP, so if their node came back online it would connect to our node. At this point we are relying on them to connect to us. This seems reasonable to me because if they were not reconnecting it would likely be because they are offline and we wouldn't be able to connect to them anyway.

If we have no out of band communication with the peer that has a private channel with us (because they initiated the channel and we have published our IP address via NodeAnnouncement), then I agree that we must rely on the node to connect back to us if they are offline. However, #10871 (comment) would fix the problem when they reconnect if they changed their IP address.

[ZZiigguurraatt]

Usually public nodes announce their IP when they reconnect so this isn't an issue. But this is a small edge case for private nodes. With knowledge of their new IP, if the connection drops again, both us and the other node can initiate the connection.

It's possible a node may want to stop accepting new public connections but keep it's existing channel peers, so it would want to change it's IP address and not include an IP address in NodeAnnouncement anymore.

Additionally, a node may need to change from a static IP to a dynamic IP and maybe they didn't want to do this, but they still had to do it so they can no longer reliably make a NodeAnnouncement with an IP address.

[ZZiigguurraatt]

However my feeling is that this doesn't give us much benefit given that we need them to be online to make a connection and if they are online they will make the connection to us. Although for correctness maybe we should add this feature.

The other use case that I did not mention is offline test networks. We may want to create multiple clones of nodes in an offline test network. In this case clone 2 of the network will get new IP addresses and we don't want node A clone 2 to try and connect to the wrong (clone 1) of node B.

[murraystewart96]

Additionally, a node may need to change from a static IP to a dynamic IP and maybe they didn't want to do this, but they still had to do it so they can no longer reliably make a NodeAnnouncement with an IP address.

I'm not sure this is an issue. If they have a dynamic IP they could still announce the new IP when they re-connect. As long as they know our IP its fine. When they reconnect we store their IP. So the two cases are

1.) they are public and they announce their IP when they re-connect to us (already implemented)
2.) they are private and we store their address when they re-connect (needs to be implemented)

We might temporarily lose the ability to initiate connections if their IP changes and they don't reconnect but i'm not sure there is anything that can be done about that.

[murraystewart96]

However my feeling is that this doesn't give us much benefit given that we need them to be online to make a connection and if they are online they will make the connection to us. Although for correctness maybe we should add this feature.

The other use case that I did not mention is offline test networks. We may want to create multiple clones of nodes in an offline test network. In this case clone 2 of the network will get new IP addresses and we don't want node A clone 2 to try and connect to the wrong (clone 1) of node B.

This point is beyond me xD. I'm not sure what the best solution here would be

[ZZiigguurraatt]

Additionally, a node may need to change from a static IP to a dynamic IP and maybe they didn't want to do this, but they still had to do it so they can no longer reliably make a NodeAnnouncement with an IP address.

I'm not sure this is an issue. If they have a dynamic IP they could still announce the new IP when they re-connect. As long as they know our IP its fine. When they reconnect we store their IP. So the two cases are

Yes, but if they have a dynamic IP now they may not want to deal with having to keep track of when the IP changes and then re-announce. I don't think there is native way in LND to do this automatically. We do have

lnd/sample-lnd.conf

Lines 79 to 88 in 036d1e0

	; A list of domains for lnd to periodically resolve, and advertise the resolved
	; IPs for the backing node. This is useful for users that only have a dynamic IP,
	; or want to expose the node at a domain.
	; Default:
	; externalhosts=
	; Example (option can be specified multiple times):
	; externalhosts=my-node-domain.com
	; externalhosts=my-second-domain.com

, but everyone is not going to necessarily want to use a third party dynamic DNS service to use that feature and we have no control of how frequent "periodically" is.

1.) they are public and they announce their IP when they re-connect to us (already implemented) 2.) they are private and we store their address when they re-connect (needs to be implemented)

We might temporarily lose the ability to initiate connections if their IP changes and they don't reconnect but i'm not sure there is anything that can be done about that.

Agreed.

[ZZiigguurraatt]

One thing we could add that we don't have now is that when the partner re-connects we don't store its new IP address.

This is a good idea and would help too. I'm wondering if that should be a separate issue though, or combined with #10870 ?

One argument against doing this is mobile nodes. Zeus running with integrated LND on a phone can roam from network to network very frequently and their IP address can then change every time they connect. So, we may not want this to be the default behavior unless we make sure we prune every address old once a new one is learned.