Latest Posts

Stealing Sats from the Lightning Network Custodial Services

Posted 1 day ago by Reckless_Satoshi

The Lightning Network (LN) is a truly groundbreaking way to move value around the globe. The number of users and LN enabled services is exponentially creeping up. Many services are opting for offering a free or fixed transaction fee, yet the real lightning network fees are neither free nor fixed. Instead, they are cheap (mostly) and variable (according to the payment route). I conducted a small research project to figure out whether the discrepancy between real routing fees and service's transaction fee can be exploited for a profit, and if so, how large the damage could be (spoiler: it is bad).

In the cover image Reckless_Satoshi is wearing a hoodie, which indicates he is up to something.

Whom did I attack? Well, here the complete list of offended services Bitfinex, OKex, Muun, WalletOfSatoshi, LNMarkets and Southxchange. If you are reading this to make a 'quick sat' I am sorry to disappoint you, I am publishing these findings only after the susceptible services have been contacted and flaws fixed :)

Cheap, but not free. A simple attack.


Simple, deposit funds into a custodial service then withdraw the funds, done. Congrats for your profit! I am sure you are thinking -"Those sats were mine anyway, right? How does this qualify as an attack?" Well, I forget to mention we also need to place a node that will be routing the payments between the custodial service and the receiving node. The routing node will collect a fee, hopefully the fee will be big enough so there is a net profit (i.e., withdrawal_fee + deposit_fee < routing_fee_collected). If a positive net return is possible, then it is just a matter of optimizing the size of the fee collected and the transaction speed rate to see how big the damage could be. It is easy to see how this attack must be feasible on any service with free withdrawal fee.


How do you place a node in the middle? Well, the sending node is in charge of selecting the route. A priori, it seems unlikely that the sender will select a very expensive route. However, there is a case when the sender will certainly have to send the payment trough our routing node. We will connect our receiving node to the Lightning Network only with a single channel to our routing node. Therefore payments, if they arrive at all, must always be relayed by ourselves.

node_structures.JPG 199 KB

Our receiving node is only connected to the Lightning Network through our routing node. Green arrows represent revenue, red arrows are costs.

In the case depicted in the figure, our routing node is directly connected to the custodial service. This is ideal to optimize the attack: the deposits have no cost, HTLCs will settle quickly, and we avoid the limitations set by other routing nodes using CircuitBreaker (payments fail when a few HTLCs are pending). If the attack is successful, having a lot of inbound liquidity from other nodes is key. The channel to the custodial service will quickly become unusable as we have stolen the liquidity to our side. Therefore, you want to desaturate it by circular rebalancing. Once we free up inbound liquidity from the custodial service, the channels to our liquidity providers will be saturated, we can chose to close those and move the profits on-chain or we could loop out (not sure which process is less costly: we are making free BTC, does it even matter?)

This is one of the simplest attacks. In fact, the only LN attack I can think of, but also I am just a newbie in the process of learning. I assume there is people out there much more capable of conducting this research. Who knows, maybe there has been sizeable loses in the past that remains undisclosed.

1. Bitfinex


Bitfinex has a fixed 100 sat withdrawal fee. However, it is obvious that some withdrawals requests might cost to route more than that. I was curious to see if withdrawals that are more expensive would be processed at all: and yes, they are processed. It is my believe, after a bit of tinkering, that Bitfinex would execute any withdrawal where payment routing fee is below 10 000 ppm (1%). I gave a try to withdraw 100K sats and collected 1000 sats in fees on the middleman node. 

Making a net profit from Bitfinex is possible (at least net positive 900 sats per deposit/withdrawal cycle), however withdrawals might quickly get halted as there is a "processing" step on their end probably rate limiting transactions. Bitfinex's API does not seem to support yet withdrawals for the symbol 'LNX' (these require an invoice instead of an address). So while it is possible to profit from Bitfinex, I didn't go the next step to script and optimize the attack. In any case, I filled out a report with their security team before making this public. Their site explicitly indicates that they might no reply to a report if they were already aware. As I received no reply I assume it's safe for this insight to go public.

2. OKex 


The fee charged by OKex seemed to be strictly equal or higher than the cost to route the payment. There is no way one could make a net profit from OKex using this attack.

3. Muun wallet

I do not know exactly how Muun works behind the scenes. It is not strictly a custodial service, but it has definitely some sort of custodial component to it. It might be maybe some sort of hybrid: possibly a parent node (named Magnetron?) with private channels to each user's wallet (but do not quote me on this). Their super easy to use LN enabled wallet allows you to withdraw all the way down to 0 sat balance without having to pay the final fee for emptying the wallet. This, in turn, allows you to collect a net positive fee for every withdrawal that empties the wallet. As this is a smartphone app and there is no available API, I did not go through the extra complexity needed to test where are the limits of cheating Muun.

4. LNMarkets


LNmarkets is possibly one of the coolest LN services out there. The use of LNURL qrcodes to login, deposit and withdraw makes it the most LNish experience out there. It truly displays what the LN is capable of, in addition, their API documentation is simply superb. Unfortunately, their effort also made it very easy for me to script and optimize the attack. Since the service had a free withdrawal fee, it was indeed profitable.

According to some quick testing, LNMarkets is willing to route to you any payment as long as the fee does not exceed 10 000 ppm (1%). The maximum deposit/withdrawal amount is 1m sats. As you can see, theoretically one could expect to make a net profit of ~10K for every deposit/withdrawal cycle. Each cycle takes around 20 second (this will greatly depend on whether the nodes are behind TOR or Clearnet). Using two threads, that makes for a profit of about ~4 million sats/hour .

At 4 m sats/hour the full outbound liquidity of LNmarkets would have been stolen in 80 hours (totaling 3.3 BTC, LNMarkets is open about their outbound liquidity on their own site). The script ran for 6 minutes, collecting about 450K sats in fees before some failsafe halted platform withdrawals for all users. About ~2 million sats were locked into the platform, for a net loss of ~1.5m sats. However, LNMarkets guys have been exceptionally cool about this and returned the sats to me. They certainly did not have to, but I appreciate it and shows they strive to build a healthy community.

LNMarkets is now charging the routing fee to the user. It is a fair and sustainable solution, although it muddies the user experience. The beautiful thing about lightning is that if the users wants often and free withdrawals, they can just open a channel to LNMarkets for the price of a single on-chain fee. 

5. Southxchange


Southxchange LN withdrawal fee is free. I tinkered a bit to find the maximum their node would be willing to pay for a successful routing. I think it was about 50 sats flat as maximum, but maybe it was higher. Even when I was withdrawing 1 sat, their payment was sent with 50 extra sats for routing. That's an effective 50 000 000 ppm (5000%) fee being collected!

It was possible to deposit 100K sats and then withdraw 1 sat a time. Of course, 50 sats is a negligible amount. However, their API works flawlessly and I noticed there was no request rate limit. I wrote a simple python script able to generate local LN invoices and submit them to the exchange to process the withdrawals. It reached top speeds of up to ~300 withdrawals per minute (200 ms per withdrawal), simply wow! That makes for ~15K sats per minute. I did not optimize further the script, as the channel was already near being maxed out (current maximum pending HTLCs for a channel is 483 and they were taking long to settle). In addition, my RaspberryPi was getting CPU limited, I believe due to encrypting/decrypting the onion packages. It would have been possible to improve the attack speed by a lot with better connection and some parallelization (more accounts / more machines / more routing nodes).

Without any further optimization, at a rate of 900K sats / hour the full outbound liquidity of Southxchange would have been depleted in ~50 days (assuming there being 10 BTC or ~1/6 of the node capacity). I stopped the script after one hour as there seemed to be no limits or failsafe whatsoever. A malicious attacker could have definitely withdrawn most liquidity in hours.

After the attack, Southxchange has opted for rate limiting withdrawals for the user (1 every 10 minutes), but they are still free. In my opinion, this is not the optimal solution. It affects the experience of legit users that need frequent withdrawals: in-and-out quickly, minimizing exposure to custionals, this is what lightning is about. Yet, this solution also fails to prevent future attacks, as you can still get around this limit with many accounts. Instead, I would suggest charging the withdrawal fee to the user: you gotta pay what things cost. If the user wants free withdrawals, they can 'go premium' by opening a channel to the exchange's node.

6. WalletOfSatoshi 

WalletOfSatoshi charges the user the exact fee for the routing. It also does hold a reserve of 0.3% balance in case of unexpected high fee. This is the most conservative take together with that of OKex, in turn making these two services the least user friendly.

If a service has free withdrawal, users are more compelled to take their BTC into self-custody between operations (it is free, why wouldn't you?). So, I am not totally sure of what I am about to say, but I have the feeling that custodial services with free transaction fees might be artificially increasing the number of transactions, therefore subsidizing nearby routing nodes. This might induce weird incentives for the creation of channels and the deployment of liquidity; hence, affecting how the lightning network grows. Yeah, sounds far-fetched, but even if tiny, there must be an impact (no idea if positive or negative impact).

Conclusions

- Although LN transaction fees are negligible, they are not zero. While lightning allows for almost free transactions it also allows for extremely fast transfers: negligible amounts add up to worrisome amounts very quickly. If you build a service where withdrawals are not rate limited nor the fee is translated to the user, you will run into problems.

- This is one of the simplest attacks anyone can think of using LN, yet surprisingly, many services are susceptible. I believe that if an actual smart and malicious actor had performed it, he could have withdrawn a big chunk of the outbound liquidity of some of these nodes.

- By attacking ourselves the LN and publicizing the findings, we make stronger the Lightning Network and its services. Maybe soon we will be reading sensationalist headlines such as "The Lightning Network has been hacked" every time a custodial service using LN is exploited. It is in our hands to prevent FUD to spread also over the amazing lightning features.

Finally yet importantly, I would like to apologize for the disruption caused to the service maintainers and thank them for their excellent sportsmanship. It has been a great deal of fun to learn how these futuristic services work.

I'm sharing code to replicate my findings on GitHub reckless-withdrawals. So far, only LNMarkets, I will not share yet Bitfinex and Southxchange as I am not 100% confident that they are exploit proof after their fix.

Let me know in the comments if there is any LN enabled service that I should test. I went for all of the big ones already but I might make a second round :)

# - Reckless_Satoshi


If you enjoyed my little research project, you can say hi by opening a channel to me or via keysend (02ce13573f6ab577088cead4379dc64f300ffbeca2ae040beee9f3541ccc4427c7) or LNURL (LNURL1DP68GURN8GHJ7MRWVF5HGUEWVDHK6TMVDE6HYMRS9ASHQ6F0WCCJ7MRWW4EXCTECXQUSW77KS4). 

Cheers!

2 Replies

Private Swaps are Here

Posted 6 days ago by LN+

The new highly requested Private 🤫 option is now available when you create a new Swap. Such private swaps are not visible on the swap index pages or on node profiles. Application to these swaps is protected by a 10 digit alphanumeric pin code.

The idea behind private swaps is to enable groups of friends to organize swaps among themselves. The creator of the swap should send the URL of the swap and the pin code provided by the LN+ interface to join the private swap to their friends over private communication channels.

Please be aware that the swap page itself is publicly visible on the web, and the opened public channels will naturally also be public on the LN gossip network. If you want the channels to be fully private and inaccessible to the public Lightning Network, you will need to open private channels and organize the swap outside of LN+. If there is a need I can implement support for fully private channel openings, but my goal is to keep LN as open as possible, while serving the needs of LN node operators.

If you spot any bugs, or have any comments, please reply here or send me an email.

Post Reply

Sup?

Posted 7 days ago by BTC Sessions

Finally home and experimenting more with my lightning node. Excited to tinker with this! Kudos on this amazing resource.

5 Replies

Posted 9 days ago by Umberta

Let me tell you why you're here. You're here because you know something. What you know you can't explain. But you feel it. You've felt it your entire life. That there's something wrong with the world. You don't know what it is but it's there, like a splinter in your mind driving you mad. It is this feeling that has brought you to me. Do you know what I'm talking about?

4 Replies

Posted 9 days ago by Umberta

1 Reply

Installing charge-lnd on GetUmbrel to automate your fee policies. Guide by The_Lorax!!

Posted 9 days ago by sackof.w4ge.com

I've just followed the guide below to automate an hourly change to my channel fee policies:

https://community.getumbrel.com/t/guide-installing-charge-lnd-in-a-docker-to-automate-your-fee-policies/2187

I've customised the config file to set 0 base fees across all channels unless there are low sats left on my side (in which case use 9999 base fees to effectively disable routing from my side). 

All channels get a proportional fee per ppm. So if there is a high balance on my side the fees will be low and as my balance decreases the fees increase.

No more manual channel config changes :)


2 Replies

Moving to a new node!

Posted 9 days ago by WolfBlitzer

Hey all, in the relatively near future (within the next month) I will be moving my channels to a new node. I will reopen each channel to everyone in my triangles. Once I do, it would be great if you would reopen your channel to me. I will post more information about the node once I have it up and running. Thank you!

1 Reply

Igniter2.0 for a more coordinated ignition of ROFs

Posted 9 days ago by hippiebro

Just checkout the repo: https://github.com/ziggie1984/miscellanous.git
For big ROFs it is sometimes the case, that when you coordinate a ignite fees propagate very slowly (gossip protocol). Therefore this script has the ability to set all fees to zero, and it is only important that the channel owners has set the fees to zero too. So you can ignite a ROF more coordinated without a lot of waiting time. The script constructs a payment and sends it through the network.

2 Replies

Open channels to key nodes helping El Salvador's efforts

Posted 11 days ago by LN+

IBEX Mercado, a major payment processor in El Salvador wrote on Twitter:
We are happy to announce that we have onboarded dozens of merchants with hundreds of store locations in El Salvador to be ready to accept LN/BTC payments. We would greatly appreciate some inbound liquidity to help us with the #hyperbitcoinization of the region.

Here are their nodes if you want to open to them (minimum 2M SAT):

OpenNode is a global payment processor that helps companies like McDonald's process their payments. Their nodes:

Official Chivo wallet's node:

As I come across other key players, I will add them to this post.

4 Replies

We're flying the El Salvadorian Flag

Posted 12 days ago by LN+

Congratulations for being the first nation state adopting Bitcoin as a legal tender!

Post Reply

Let's make a new world...

Posted 13 days ago by deej

...where we control our own financial destiny by just using good old immutable MATH!

1 Reply