A Guide for Situations Where Your Lightning Close-Transaction Fails to Close the Channel

Posted 10 months ago by HODLmeTight

We've all been been there, a channel gets closed, no-matter yet if it was a cooperative or force-closed, but somehow the close transaction hangs around and doesn't get confirmed. You want to clear the house, or need the pending onchain sats, but wonder how to go about it? Follow this guide and you'll hopefully find remedy.

But before, read this article from LND. Then read it again. Once you think you understand it, read it again. It's not super cohesive, since a number of things from the top need to be understood to grasp the section Channel closing, so ensure you spend some time in there.

Situation

This is written for LND, but similar methodologies are applicable for CLN. Eclair folks know their way around anyway. You identified a channel-close, either in one of the GUIs you use, or from a Bot notification. But it doesn't confirm. First thing to do is SSH into your node via ssh user@node-ip and get an overview of all pending (open, and close) with lncli pendingchannels.

For umbrel ☂️, you need to jump through some loops for the command lncli . You need to add this whole string before: ~/umbrel/scripts/app compose lightning exec -it lnd lncli. To simplify things, create a temporary alias with alias lncli="/home/umbrel/umbrel/scripts/app compose lightning exec -it lnd lncli", then all of the above commands should work.

{  
  "channel": {  
	"remote_node_pub": "123456789889786857463636288291929458767688493999392324",  
	"channel_point": "dhksfkeh8458734kldjfksd39393932033fkk3003022244:1",  
	"capacity": "10000000",  
	"local_balance": "9999711",  
	"remote_balance": "70137",  
	"local_chan_reserve_sat": "10053",  
	"remote_chan_reserve_sat": "10053",  
	"initiator": "INITIATOR_REMOTE",  
	"commitment_type": "ANCHORS",  
	"num_forwarding_packages": "2",  
	"chan_status_flags": "ChanStatusBorked|ChanStatusCommitBroadcasted|ChanStatusLocalCloseInitiator"  
  },  
  "limbo_balance": "9999711",  
  "commitments": {  
	"local_txid": "eeof599490930002dcc0db3bfa789eb4afbd89a37794a4c49a5f529ddb1b23e85b3",  
	"remote_txid": "ta48849fkkek39947uuc80371ec96a88297465821ee79ff0b85f8624f83a7455f1cc",  
	"remote_pending_txid": "",  
	"local_commit_fee_sat": "2510",  
	"remote_commit_fee_sat": "2510",  
	"remote_pending_commit_fee_sat": "0"  
  },  
"closing_txid": "eeof599490930002dcc0db3bfa789eb4afbd89a37794a4c49a5f529ddb1b23e85b3"

What we see here is that our LND sees that channel as borked, we have a an opening_tx (channel_point:vout), a local_txand a remote_tx. What's usually the next step, we wonder why it's not closing, so we copy the

"closing_txid": "eeof599490930002dcc0db3bfa789eb4afbd89a37794a4c49a5f529ddb1b23e85b3"

and copy it into our favorite mempool tool. Just for guiding sake, we check with mempool.space, and now it's the time to split into two scenarios:

Scenario 1: mempool.space shows the closing-tx, but it's unconfirmed and the fee is too low

What many runners don't know, mempool.space runs with a mempool-size of 2GB, while most standard node-systems run with just 300MB. This is usually enough, but not in high fee environments. So let's check if your local mempool has it:

Similar to lncli with umbrel ☂️ you need to follow a similar approach for bitcoin-cli: ~/umbrel/scripts/app compose bitcoin exec -it bitcoind bitcoin-cli for the one off-command, or alias bitcoin-cli="~/umbrel/scripts/app compose bitcoin exec -it bitcoind bitcoin-cli"

bitcoin-cli getrawtransaction eeof599490930002dcc0db3bfa789eb4afbd89a37794a4c49a5f529ddb1b23e85b3
error code: -5
error message:
No such mempool or blockchain transaction. Use gettransaction for wallet transactions.

In this case your mempool probably purged the transaction, or LND never published it due to it's low commitment fee. Let's jump to the next section to see what we can do about it.

Scenario 2: mempool.space shows Transaction not found

Okay, we can assume that your closing-tx probably never hit the public-mempool, we can cut things short here and move to the next section, but just to be safe, let's check if our local bitcoin-mempool has it:

bitcoin-cli getrawtransaction eeof599490930002dcc0db3bfa789eb4afbd89a37794a4c49a5f529ddb1b23e85b3
error code: -5
error message:
No such mempool or blockchain transaction. Use gettransaction for wallet transactions.

Complication

Why are there transactions of channel-closes with too low fee? Can't LND figure out what fee is necessary and get things just out there? I want my funds!

Let's assume best intend, no-one built that protocol to make things intentionally complicated. But since you have two parties involved in the process, and we should assume no-trust between those, we need to install processes to keep things fair, balanced and transparent. Read up on the HTLC-detail guide from Elle Mouton to make yourself more familiar with the details.

In short, we have 2 (out of 3) possible ways for a channel-close:

cooperative-close, where both parties are online, one of the two sends a channel-close signal, and then both nodes get to court and negotiate a closing price. Since only the opener pays for the closing-cost, it's imminent it's not sky high. There are settings in lnd.conf like coop-close-target-confs=100 which help to keep things rational from your side if the other side triggered the close.
force- or uncooperative-close, where either one of the two parties is unresponsive / offline, or other protocol / node-OS disagreements happen (like expiring HTLCs). In either case one of the two parties force-close, there is no court-negotiation, but based on the last channel_commit agreement the two nodes handshaken on, will be used for the closing-fee. Those agreements are also based on a few lnd.conf settings, like max-commit-fee-rate-anchors and bitcoind.estimatemode.

Wonder what your channel-peers current commit-fee is? And whether they use anchors or static channel-fee commitments? Check out this bash-script, among other Node-Tools available as open-source.

Either have their intricacies, but when the channel-close is not in your local mempool, or too low fee in the public mempool, you can't get the channel closed and the onchain-sats stay locked away.

Proposed Solutions

We won't outline every possible scenario, but a couple of most happening ones. Some will work, others not. To avoid this article becoming a master-class, let's go through a couple of those which are doing no-harm, but helping you worse case to get some more information what's happening.

For each of those attempts, prepare a second terminal window to your node, and filter (grep) the channel-point and pubkey of the channel, so you'll be more informed after of what's happening. This terminal will monitor your lnd.log and show only entries where either the Pubkey of your to be closed channel-peer, or your mutual channel-point is triggered.

sudo tail -f .lnd/logs/bitcoin/mainnet/lnd.log | grep -E 'PUBKEYOFCHANNELPARTNER|CHANNELPOINT:VOUT'

While Terminal 2 stays open, let's move to Terminal Window 1 for the next couple of commands. We'll try one after the other, and this guide will be verbose if there's risk or sat-spending involved. It's sorted from lowest risk / spend to highest, top down.

1. Reconnect

Many coop-close attempts fail, because when both nodes go to court, there is a possibility to not find consensus on the closing fee. Let's see if this is the reason, you'll see entries like commit-fee proposal staggering up, and whether you'll find consensus lncli disconnect PUBKEY && lncli connect PUBKEY@ADDRESS:PORT If the closing attempts fail, you'll see that each time, the delta between your node and your channel-peer will get closer. Eventually it'll work, if not, check your lnd.conf settings and eg change bitcoind.estimatemode=ECONOMICAL to bitcoind.estimatemode=CONSERVATIVE.

2. Restart LND

Sometimes closing attempts miss a broadcast due to a missing connection to bitcoind. Restarting LND rebroadcasts every pending channel-close and sweeps, so give this a shot and check your terminal 2 while LND is taking it's time to come back up. Really, you need some patience here.

3. Reinitiate coop-close (and force-close)

Try to close the channel again, in rare cases this would solve something which 1. didn't: lncli closechannel --chan_point CHANPOINT:VOUT --sat_per_vbyte XX. This may offer some more insight in your Terminal 2 window. You can do the same for the force-close, just need to adjust the command slightly. For the record, I never had this situation, but it's not doing any harm in case you know it's a force-close anyway: lncli closechannel --chan_point CHANPOINT:VOUT --force. We can omit the target sat/vbyte setting since force-closes use the last agreed commit-fee of both you and your peer.

4. Increase local Mempool size

This solves for a couple of things, both Scenario 1 and Scenario 2 above, at least as a preparation. Open your bitcoind-configuration with sudo nano .bitcoin/bitcoin.conf to lookup and edit the following entry. Take it to 600 MB to capture all tx of the past two weeks, or up to 2000, (2GB) if not sufficient:

# Increase Mempool size
maxmempool=600

Restart bitcoind (typically sudo systemctl restart bitcoind to take this setting into effect. And either LND already restarted by itself, or trigger a manual restart with sudo systemctl restart lnd. Watch Terminal 2, to see whether your 300MB previous mempool was too limited to accept your LND-closing. With 2GB, that limitation is lifted (a little) and potentially allows your LND to broadcast the closing-tx.

5. Manually broadcast the closing-transaction

Eventually 4. allows your local mempool to pick up the closing transaction. Sometimes it doesn't. Or your mempool has it, but the public mempool didn't. Without either your bitcoind node, or the public mempools knowing about this tx, your channel can't get closed. In this event, let's look at how to broadcast it manually and see what response we get in both Terminals.

Local Mempool check: Follow Scenario 1.
If it's a No, does mempool.space have it? Then open the tx there, and grab the RAW-tx by appending the txid here: https://mempool.space/api/tx/[TXID]/hex. Copy the whole long string, and execute the local broadcast in your Terminal 1 with lncli wallet publishtx RAWTX.
If that's a No too, check your LND, it should have it (only in rare circumstances, like SCBs, LND doesn't have the raw-tx: lncli listchaintxns --start_height 818181 --end_height -1. Substitute 818181 with the current mempool height to filter the transaction output showing only not confirmed pending transactions (utxos). You'll get the raw-tx there, and can use it with lncli wallet publishtx RAWTX as above.
If your local Mempool has it, but mempool.space doesn't, there is no harm broadcasting it there as well. Follow Scenario 1 to retrieve the raw-txid from your local mempool, and copy it in here.

6. Increase the closing-fee

So mempools know about your closing-tx, but it's way too low. Let's assume it hangs at 10sat/vbyte (like many pending closing-tx these days), but let's also assume we won't see <30sat/vbyte anytime soon. Because you'd always have the option to wait for 10sats, your tx will eventually confirm. But you don't want to wait, then you'll need to pay up. Disclaimer: This one is costing you. No matter who the opener of the channel was (who always pays for both un- and cooperative-closings), the next step costs You. So be concious about it.

The LND Guide linked above has a succinct closing channels section. And since you inhaled that guide, you'll know the next steps:

For Coop-Closes, you can lncli wallet bumpfee -h to make yourself familiar with the options. It's pretty self-explanatory, lncli wallet bumpfee --conf_target 6 CHANPOINT:VOUT will ask your local mempool for a sat/vbyte target and set the new closing fee. With lncli wallet bumpfee --sat_per_vbyte 36 CHANPOINT:VOUT, you can set a confident 36 sats/vbyte yourself. Watch Terminal 2 and see what lnd.log tells you.
For Force-Closes, you need a so called CPFP (Child Pays For Parent) transaction, which will become a child of your existing closing fee, and at the same time expensive enough, that both parent + child and their fees will be attractive enough for a miner to pick up and mine it. ~~You can either do some pre-calc~~ ~~with a helper script, or try current mempool medium fee x 2~~. Not a problem if you don't start high enough, you can follow step 1. above here with this new child-transaction and bump it's fee with RBF any time later: lncli wallet bumpclosefee -h for the options:

lncli wallet bumpclosefee --conf_target 6 CHANPOINT:VOUT will ask your local mempool for a sat/vbyte target and set the fee for the child-tx.
lncli wallet bumpclosefee --sat_per_vbyte 100 CHANPOINT:VOUT will target 100vByte for the combined tx, so your child will have a much higher (probably 2x) effective rate for both your parent and your child to be mined at a 100.

Note that none of this works if your local mempool doesn't have that txid. So follow steps 1-5 before trying this one. Now but if it worked, you'll be prompted with a new txid for your child, which you can later check in your local mempool, and ideally it'll land in the public mempools to be mined as well. If it ended up too low, bump it with RBF (replace by fee) described in point 1, but targeting your child.

7. Go out of band

Just for completion, there are various miners who offer their mining service for your tx for a hefty cost. This sometimes might be necessary, eg for your channels where the other side force-closed on you, but you only have a static channel-commitment. If your peer is unresponsive or not cooperative to do a CPFP on their side, you'll wait until forever. Check your favorite search engine, ViaBTC or mempool.space for their acceleration services.

Hope you enjoyed this article. Please do share feedback and suggestions for improvement. If this guide was of any help, I'd appreciate if you share the article with others, give me a follow on X or nostr, perhaps even donating some sats to hakuna@getalby.com. I'm also always grateful for incoming channels to my node: HODLmeTight.