Chillycalendar and Chillycalendar 2 have been HACKED, funds have been STOLEN and both hosts are being taken down
Posted about 1 month ago by CHILLYCALENDAR
Dear Fellow Lightning Network Node Operators,
It is with great sadness that I announce that my two R&D Bitcoin Lightning Network Nodes CHILLYCALENDAR and CHILLYCALENDAR2 have been hacked and I am taking them offline until further notice.
This morning I woke up to several emails from node operators from this website asking me why I had closed channels to their nodes. I had not closed any channels. I attempted to login to this lightningnetwork.plus website and my password didn't work. This was the first sign that something was seriously wrong. So I requested a new password and I was then able to login and start responding to node operators who had messaged me.
What do I suspect happened?
I suspect that I stupidly used the same password on my main web host server and my two bitcoin nodes as the password I used on this website. The hacker I suspect may have somehow managed to gain my password from this website and then used that password to login to my primary web host where they entered these commands without cleaning up after themselves:
image.png109 KB The hacker was looking for bitcoin wallets on my primary web host. They also looked through KYC Data (I have no real world active users KYC data so they got nothing there. My systems are primarily for R&D and proof of concept purposes).
I ran the last command to see if I could find where the hacker came from:
image.png5.79 KBimage.png70.7 KB
The hackers IP address is: 185.213.155.215
LN+ Alan - you might want to check your servers and make sure this IP address has not infiltrated your systems and to put a block on it anyway as a precaution.
Fellow node operators - I suggest changing your password you use to access the LN+ website as a precaution and do not use the same password on this website or others that you use on your bitcoin nodes. I was not cautious enough and I believe I used the same password on this website as my primary nodes. I'm kicking myself for my stupidity. If anyone else wants to kick/slap me do feel free I deserve it. In my defence - I have been very distracted with my health which has been very poor due to two torn meniscus in both knees which I have been doing a lot of rehab on thus I have been away a lot from my servers and highly distracted learning to walk again in the local swimming pool.
After finishing looking around on my primary web host the hacker gained access to my primary node CHILLYCALENDAR. From there they somehow managed to gain access to my BTCPay Server instance without generating any history in the shell. They turned off 2FA and then they closed all my channels I had open to other nodes. They did the same thing on my secondary node CHILLYCALENDAR2 and they also changed my password on this server of my BTCPay Server instance. After closing out all my channels they then proceeded to steal approx $10,000 AUD of Bitcoin funds I had on both the servers. These funds were made up of on chain funds and funds in channels I had initiated to other nodes via my RTL instance on my BTCPay Server. Luckily for me I had recently removed a lot of funds from my nodes in order to help my family so my losses were limited and it could have been a lot worse.
Whilst the attack itself was on the whole not sophisticated there were parts of it that were. Managing to close off channels and take funds without leaving much of a trace via my BTCPay Server log files for example. It would seem that the hacker was only interested in stealing Bitcoin. Whilst the hacker did gain access to some of my other nodes unrelated to Bitcoin they didn't do anything on those nodes other than take a look around.
I have gone through and changed all my passwords on all of my hosts and my personal passwords for everything else as well.
Obviously I blame myself for my poor security practices. I should have and I did know better.
Thank you to my fellow node operators who alerted me this morning to issues surrounding closed channels. It's unlikely I will be able to respond to all of you personally but thank you for messaging me and I'm very sorry for what has happened. Feel free to message me if you have any information related to this hack that may help other node operators or just to slap me around a bit for my stupidity.
Logs on my primary web host indicate the hacker has been through my email as well unfortunately. I have changed this password as well.
Both CHILLYCALENDAR and CHILLYCALENDAR2 were R&D platforms. The purpose of which was to run bitcoin lightning network nodes and see how far I could get as a sole operator. CHILLYCALENDAR was the 2nd highest ranked node in all of Australia and had processed / forwarded over $1 million worth of lightning transactions and i'm very proud to have achieved this working completely on my own. I have been running lightning nodes since 2021 and I almost got to the 4 year mark of running these servers in the wild without any incidents. This work has enabled me to create a lot of documentation and obviously I have learned a great deal.
Will I be back? I'm not sure yet. I'm still processing everything that has happened. But rest assured that the output of everything I have done, learned etc will go into something in the future if not as a sole operator then likely as part of a dedicated team working for a larger corporation that has the resources and people required to enable me not to have to do literally everything on my own and it has helped the bitcoin lightning network to mature having my nodes online since 2021.
Kind Regards,
Clarke
Clarke Towson, BCMS (Bachelor of Computer & Mathematical Science)
CEO
INTJ Billing
m: +61 432 359 166
a: 7 Cullen Court Spotswood Victoria 3015 AUSTRALIA
This is terrible. So sorry! Thanks for providing the details of the attack. It's very considerate of you to give a heads up!
Your post got my blood pumping for sure. I immediately checked as deep as I could, and so far there is no obvious trace of an attack on LN+ servers directly. But, I will keep trying to think of ways to investigate more, and will also monitor closely the servers.
I did find info on the IP you provided. Here are the events I can reconstruct:
Tries to sign in unsuccessfully with the provided IP at 2025-02-20T14:25.
Asks for a password reset email with the provided IP.
Visits the about page with the provided IP.
The provided IP doesn't have any activity going forward, so the next steps are somewhat of a speculation.
The attacker accesses your email to get the password reset link with the token. This strongly suggests that at this point the attacker has gained access to your email, and I'm speculating that he's logging in to all potentially interesting services. Check your email archive or trash if it's still not emptied for evidence of activity you didn't do.
Visits the password reset page from a new IP. This is likely based timeline and activity. (I can send IP privately.)
From the new IP resets the password and logs in.
From the new IP visits a whole bunch of random pages, possibly trying to find something useful with a bot. The activity is relatively slow, so it's not picked up as a bot. The activity also doesn't look like a human, because it visits pages that are strange like this. After a many hours the activity seizes at 2025-02-21T04:55.
From public sources we know the IP you provided is a VPN server in Germany. The other new IP is likely a proxy server in the US.
From what we know, we can speculate that somehow the attacker managed to get into your email and through that into other services like LN+. For what its worth, your reported password reuse most likely didn't cause the issue.
If they sent out the funds on-chain from your node, you should be able to track the funds. I think you can file a police report and also report the address to all exchanges.
Btw, even if somebody gained access to the DB for passwords we use security best practices, bcrypt which includessalting, making brute-force attacks extremely difficult. If you asked me to give you your password I would not be able to.
I will keep thinking about what evidence I can dig up to help you.
Thank you Alan for looking into this closely for me very much appreciated. I have sent an email to the web host provider for that IP Address detailing the hack. I will file a police report with German authorities as well.
Today I have gone through and changed all my passwords for everything including my email password.
I will provide more information over the next few days when I can after I have completed my full analysis. If the IP address is real then the person should be pretty easy to catch as the hosting provider will have their contact details. It's Bitcoin as well that they have stolen so everything is on chain. I have all the transactions in a log file so the funds will be easy to trace. It's amazing that people steal bitcoin given it's open public blockchain it's a full record of their illegal activities.
My main concern at this stage is the impact this may have on other node operators and your website. It's likely that it's only me that has been directly affected as the attacker targeted my very limited outbound liquidity on my BTCPay Server nodes. I keep that liquidity quite low but still it was enough for an attacker to go after. My hope is that nodes with channels opened to me have not been exposed in any way. I had a lot of other nodes who I connected with via LN+ with open channels to my nodes (over 100) and I had a lot of inbound liquidity so there were a lot of forced closures that took place when the attacker closed all the channels. I am responding as best I can to node operators who are reaching out to me.
From my server logs I see the attacker accessed directly the wallet.dat file on my BTCPay Servers. As the attacker gained root access they were able to remove the 2FA I had setup on BTCPay and they changed my BTCPay passwords as well. The servers are currently shut down.
If I rebuild future nodes will allow root login only from 1 secure device on my premises under my full control. I will also implement other security measures. I thought I had gone far enough but I was wrong.
